-- Logs begin at Tue 2022-09-13 12:13:25 UTC, end at Tue 2022-09-13 13:26:37 UTC. -- Sep 13 12:44:39.521056 dv-cinder-801354-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 12:44:39.531716 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 12:44:39 2022] *** Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: nodename: dv-cinder-801354-10-iscsi Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: machine: x86_64 Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: clock source: unix Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: pcre jit disabled Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: detected number of CPU cores: 4 Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: current working directory: / Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: detected binary path: /usr/bin/uwsgi-core Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: your processes number limit is 47680 Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: your memory page size is 4096 bytes Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: detected max file descriptor number: 2048 Sep 13 12:44:39.532847 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: lock engine: pthread robust mutexes Sep 13 12:44:39.533607 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: thunder lock: enabled Sep 13 12:44:39.533607 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 12:44:39.533607 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 12:44:39.572979 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: Python main interpreter initialized at 0x55bff6aff9c0 Sep 13 12:44:39.572979 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: python threads support enabled Sep 13 12:44:39.572979 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: your server socket listen backlog is limited to 100 connections Sep 13 12:44:39.572979 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: your mercy for graceful operations on workers is 90 seconds Sep 13 12:44:39.573309 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 12:44:39.573390 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: *** Operational MODE: preforking *** Sep 13 12:44:39.573645 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: *** uWSGI is running in multiple interpreter mode *** Sep 13 12:44:39.573645 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: spawned uWSGI master process (pid: 81565) Sep 13 12:44:39.573857 dv-cinder-801354-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 12:44:39.574403 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: spawned uWSGI worker 1 (pid: 81577, cores: 1) Sep 13 12:44:39.574748 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: spawned uWSGI worker 2 (pid: 81578, cores: 1) Sep 13 12:44:39.574897 dv-cinder-801354-10-iscsi devstack@keystone.service[81565]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 12:44:43.932608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 12:44:43.932608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: from cryptography.utils import int_from_bytes Sep 13 12:44:43.933380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 12:44:43.933380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: from cryptography.utils import int_from_bytes Sep 13 12:44:43.933518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 12:44:43.933518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: from cryptography.utils import int_from_bytes Sep 13 12:44:43.933518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 12:44:43.933518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: from cryptography.utils import int_from_bytes Sep 13 12:44:43.983072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 12:44:43.983243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 12:44:43.983443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] command line args: [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 12:44:43.983712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 12:44:43.983796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 12:44:43.983875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ================================================================================ {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 12:44:43.984020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 12:44:43.984139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] admin_token = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.984224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] command line args: [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 12:44:43.984387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] config_dir = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.984471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 12:44:43.984579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.984657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ================================================================================ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 12:44:43.984765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] config_source = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.984844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] admin_token = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.984949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] debug = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.985066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config_dir = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.985142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.985446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.985446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] config_source = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.985638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.985732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] debug = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.985922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] list_limit = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] log_config_append = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.986965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] log_dir = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] log_file = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_config_append = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_dir = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.987917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988029 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.988858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.989830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.990994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.991114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.991226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] publish_errors = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.991405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.991597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.991721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.991832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] publish_errors = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.991942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.992099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.992219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.992336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.992453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.992584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.992707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] standard_threads = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.992825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.993564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.993687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.993819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.993819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] use_journal = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] standard_threads = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] use_json = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] use_stderr = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] use_syslog = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.994931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.995047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_journal = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.995159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.995269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_json = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.995269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_stderr = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.995483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.995592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] use_syslog = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.995701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.995816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 12:44:43.995927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.996114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.996234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.996350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.996466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.996577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.996686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.996905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.997918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.998950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:43.999903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.000945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.001870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.002083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.002083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.002337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.002448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.002568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.002818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.002942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003845 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.003956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.004864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.005672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.009566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.009566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.009828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.009929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.010978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.011974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.012888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013279 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.013827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.014831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.015916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.016950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.017931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.018974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.019933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] auth.external = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.020961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021031 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] auth.password = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.external = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] auth.token = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.021993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.password = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] auth.token = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] credential.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.022897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.023050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.023129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.023335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.023721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.023721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.023876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.023950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.024886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.025886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.026888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.027872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.028986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.029983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.030954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.031984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.032939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.033902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.035973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.036864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.037979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.038930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.039009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.039164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.039251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.039406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.039519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.039639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.040302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.040383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.040565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.040691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.040969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.041991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.042960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.043934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.044890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.045863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.046876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.047991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.048952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.049932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.050958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051669 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.051948 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.052889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.053868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] role.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] role.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.054994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.055095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.055232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.055312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.056997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.057144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.057249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.057380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.057499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.057630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.057793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.057912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.058033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.058164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.058283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.058414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.058503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.058754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.058833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.059907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.060990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.061984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.062149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.062229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.064583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.064701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.064820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.064905 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.064978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.065933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.066946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.connection = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.067972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.068926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.069897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.070829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071827 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.071916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.072077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.072157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81578) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 12:44:44.072318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.072565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.072801 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.073044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.073282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 12:44:44.073454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 12:44:44.102130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.103010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.103245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.103804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.104044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.104278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.111718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.112697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.145587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.145805 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.145928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.146038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.206368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.206608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.255603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.255838 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.256271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.256501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.419936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.420208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.420344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.420476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.420603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.420756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.420885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.421027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.421154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.421279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.421279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.421520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.448469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.448656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.448791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.448918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.449059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.449198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.449322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.449444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.449566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.449698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.449890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.450035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.450035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.450272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.461449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.463322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.474257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.474493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.474725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.474951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81578) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.475837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.476089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.476214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 12:44:44.499193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.499371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.499502 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 12:44:44.499626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 12:44:44.499750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.499873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.500004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 12:44:44.500128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 12:44:44.500263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.500387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.500510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.500632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.500763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.500886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.501008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.501138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.510590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.510742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.510894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.511029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.511154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.511282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.511409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.511544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.518367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.518501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.521232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.521421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.521562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.521718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.521898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522568 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.522953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.523089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.523171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.523370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.523451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.523646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.523736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.523924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.524003 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.524380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.547069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.547237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.548528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.548667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.548819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.548944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.553433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.553639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.553790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.553955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.554082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.554243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.561008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.561151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.561309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.561447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.566939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.567200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.567283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.567497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.567578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.567787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.567876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568895 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.568974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.569170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.584653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.584858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.584943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.585155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.588811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.589251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.592032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.592336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.592480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.600579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.600689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.600782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.600865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.600939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.603462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.603762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.603962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.604058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.604268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.604350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.604557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.604638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.604855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.604950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.605141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.605425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.621036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.621324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.621594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.621891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.622018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.622312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.622596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.622866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.628645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.629620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.630211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.631203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.631742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.632023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.632734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.633015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.634742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.635040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.635328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.635614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.635783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.636013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.636096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.636354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.636642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.637047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.647073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.647358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.647644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.647930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.648108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.648318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.648403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.648625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.648707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.648910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.649000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.649350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.649832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.650123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.664156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.666023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.667610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.668925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.670500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.671823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.671935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.673265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.673396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.673684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.674119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.674772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.675187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.675503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.681022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.682489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.682584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.683956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.685402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.685831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.686127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.686414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.686843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.687247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.687540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.687824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.694134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.694435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.694724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.695597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.695895 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.696208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.701619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81578) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.701933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.702233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.702517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.702798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.703062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 12:44:44.703151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.703598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.703678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.703877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.703963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.704151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.704240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.704423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.704502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.704697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.704778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.704973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.705055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.705256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.705387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.705518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81578) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.705608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.705815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.706102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.706382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.706676 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.710401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 12:44:44.739898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bff6aff9c0 pid: 81578 (default app) Sep 13 12:44:44.741433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bff6aff9c0 pid: 81577 (default app) Sep 13 12:44:44.744113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d2c4-60f0-4478-8150-7ec7312b7a05 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:44.744403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d2c4-60f0-4478-8150-7ec7312b7a05 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:44.744678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d2c4-60f0-4478-8150-7ec7312b7a05 None None] PATH_INFO: `/v3` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:44.748840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1/1] 10.222.0.72 () {50 vars in 791 bytes} [Tue Sep 13 12:44:44 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:44:52.503102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cab44-4789-4385-96c2-6a247d22f6b7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:52.503612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cab44-4789-4385-96c2-6a247d22f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:52.503612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cab44-4789-4385-96c2-6a247d22f6b7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:52.507885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1/2] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:44:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:44:52.514001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:52.514423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:52.514541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:52.584240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: INFO dbcounter [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] Registered counter for database keystone Sep 13 12:44:52.603064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG oslo_db.sqlalchemy.engines [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81578) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 12:44:52.609212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writer thread running {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 12:44:52.799167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG passlib.handlers.bcrypt [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81578) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 12:44:52.799167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG passlib.handlers.bcrypt [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81578) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 12:44:52.849060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:44:52.850696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:44:52.976523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 2/3] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:44:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 12:44:53.477813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cab44-4789-4385-96c2-6a247d22f6b7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:53.547595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: INFO dbcounter [None req-c25cab44-4789-4385-96c2-6a247d22f6b7 None None] Registered counter for database keystone Sep 13 12:44:53.569254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG oslo_db.sqlalchemy.engines [None req-c25cab44-4789-4385-96c2-6a247d22f6b7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81577) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 12:44:53.575539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writer thread running {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 12:44:53.695993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cab44-4789-4385-96c2-6a247d22f6b7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:53.700221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf5eca-bd5b-46f9-b115-0a8bfd244516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:53.700846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf5eca-bd5b-46f9-b115-0a8bfd244516 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:53.701131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf5eca-bd5b-46f9-b115-0a8bfd244516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:53.701403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf5eca-bd5b-46f9-b115-0a8bfd244516 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:53.709772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf5eca-bd5b-46f9-b115-0a8bfd244516 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:53.761967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cecf5eca-bd5b-46f9-b115-0a8bfd244516 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 12:44:53.762257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf5eca-bd5b-46f9-b115-0a8bfd244516 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:53.771479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 2/4] 10.222.0.72 () {58 vars in 1324 bytes} [Tue Sep 13 12:44:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 12:44:53.778599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:53.795237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d41d3c-11d4-403a-bbcc-444aa69a5ea1 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:53.799580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f2c3f-3bc6-4705-a4c2-63fd895ff1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:53.800682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f2c3f-3bc6-4705-a4c2-63fd895ff1af admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:53.800951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f2c3f-3bc6-4705-a4c2-63fd895ff1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:53.801230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f2c3f-3bc6-4705-a4c2-63fd895ff1af admin admin] PATH_INFO: `/v3/projects/cdd5b4a37f1d439babc288358ccc8191` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:53.802808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f2c3f-3bc6-4705-a4c2-63fd895ff1af admin admin] RBAC: Authorizing `identity:get_project(project_id=cdd5b4a37f1d439babc288358ccc8191)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:53.855450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9c2f2c3f-3bc6-4705-a4c2-63fd895ff1af admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 12:44:53.855787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f2c3f-3bc6-4705-a4c2-63fd895ff1af admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:53.860478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 3/5] 10.222.0.72 () {56 vars in 1207 bytes} [Tue Sep 13 12:44:53 2022] GET /identity/v3/projects/cdd5b4a37f1d439babc288358ccc8191 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:44:55.988934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ea02-c5f8-4e3f-87df-a58f0b269960 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:55.989530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ea02-c5f8-4e3f-87df-a58f0b269960 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:55.989530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659ea02-c5f8-4e3f-87df-a58f0b269960 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:55.993608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 3/6] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:44:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:44:55.999240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ca61-f7cf-4fc0-8815-5453aefa3730 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:56.000657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ca61-f7cf-4fc0-8815-5453aefa3730 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:56.000657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ca61-f7cf-4fc0-8815-5453aefa3730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:56.020527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1969ca61-f7cf-4fc0-8815-5453aefa3730 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:44:56.022083 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1969ca61-f7cf-4fc0-8815-5453aefa3730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:44:56.030562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 4/7] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:44:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 13 12:44:56.309407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659ea02-c5f8-4e3f-87df-a58f0b269960 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:56.321849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659ea02-c5f8-4e3f-87df-a58f0b269960 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:56.326022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c27826-1f6b-41c0-a425-869ccaa3ffe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:56.326619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c27826-1f6b-41c0-a425-869ccaa3ffe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:56.326902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c27826-1f6b-41c0-a425-869ccaa3ffe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:56.327176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c27826-1f6b-41c0-a425-869ccaa3ffe8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:56.335012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c27826-1f6b-41c0-a425-869ccaa3ffe8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:56.337454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c27826-1f6b-41c0-a425-869ccaa3ffe8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:56.346093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 4/8] 10.222.0.72 () {58 vars in 1324 bytes} [Tue Sep 13 12:44:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 12:44:56.351381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ca61-f7cf-4fc0-8815-5453aefa3730 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:56.360945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ca61-f7cf-4fc0-8815-5453aefa3730 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:56.365267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa3c03-e3cc-4c1d-9e46-7a8dfbc46728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:56.365932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3c03-e3cc-4c1d-9e46-7a8dfbc46728 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:56.366217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3c03-e3cc-4c1d-9e46-7a8dfbc46728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:56.366491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa3c03-e3cc-4c1d-9e46-7a8dfbc46728 admin admin] PATH_INFO: `/v3/users/259c064f989f4405a7a84f397fec6223` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:56.367870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa3c03-e3cc-4c1d-9e46-7a8dfbc46728 admin admin] RBAC: Authorizing `identity:get_user(user_id=259c064f989f4405a7a84f397fec6223)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:56.369689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa3c03-e3cc-4c1d-9e46-7a8dfbc46728 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:56.371245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 5/9] 10.222.0.72 () {56 vars in 1198 bytes} [Tue Sep 13 12:44:56 2022] GET /identity/v3/users/259c064f989f4405a7a84f397fec6223 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:44:58.558918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5a11-81ee-40e5-ad51-e24560653f0c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:58.559455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5a11-81ee-40e5-ad51-e24560653f0c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:58.559455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d5a11-81ee-40e5-ad51-e24560653f0c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:58.560773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 5/10] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:44:58.565922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b367b-babc-46d5-aa14-5163d2a203bf None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:58.566337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b367b-babc-46d5-aa14-5163d2a203bf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:58.566468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b367b-babc-46d5-aa14-5163d2a203bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:58.584834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-755b367b-babc-46d5-aa14-5163d2a203bf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:44:58.586023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-755b367b-babc-46d5-aa14-5163d2a203bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:44:58.634290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fa5c5-bffc-451e-b594-2e2a6960890f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:58.634503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fa5c5-bffc-451e-b594-2e2a6960890f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:58.634750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fa5c5-bffc-451e-b594-2e2a6960890f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:58.636062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 6/11] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:44:58.641240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:58.641518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:58.641974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:58.667405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 6/12] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:44:58.685688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG passlib.handlers.bcrypt [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81577) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 12:44:58.686101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG passlib.handlers.bcrypt [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81577) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 12:44:58.735500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:44:58.737060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:44:58.833396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 7/13] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:44:58.943604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b367b-babc-46d5-aa14-5163d2a203bf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:58.969146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:44:59.048173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b367b-babc-46d5-aa14-5163d2a203bf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.081846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.082473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.082748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.083018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.094890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:44:59.103302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:44:59.104357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:44:59.105205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 7/14] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:44:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:44:59.110189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:59.111604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:59.192845 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:44:59.212200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5fd63-c041-459a-bb68-42a16381b75f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.213513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d610d1-5874-4e6b-99f0-4a144057496c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.215846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:44:59.246319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.246921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.247204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.247467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.250566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.251425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.251706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.251979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.252810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:44:59.253113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:59.254623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:59.268011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 8/15] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:44:59.274319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:59.313040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:59.314662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:59.373043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6a804-2e49-4dd6-a9a6-4211575a5772 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.392300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:44:59.405907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.406768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.407052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.407327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] PATH_INFO: `/v3/users/259c064f989f4405a7a84f397fec6223` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.408815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] RBAC: Authorizing `identity:get_user(user_id=259c064f989f4405a7a84f397fec6223)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:59.410338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:59.410835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 8/16] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 12:44:59.411892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 9/17] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:44:59 2022] GET /identity/v3/users/259c064f989f4405a7a84f397fec6223 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:44:59.424520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:59.424658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:59.453980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:44:59.515664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39714c-dce4-487b-8439-137a0317782e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.518640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e17a83-df98-4b28-adaf-ca081b326f9d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.548630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.549257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.549536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.549836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.551769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846103-536d-406b-a438-f0e4c273381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.552392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846103-536d-406b-a438-f0e4c273381e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.552670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846103-536d-406b-a438-f0e4c273381e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.552942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846103-536d-406b-a438-f0e4c273381e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.554074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846103-536d-406b-a438-f0e4c273381e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:59.556181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846103-536d-406b-a438-f0e4c273381e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:59.557708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 10/18] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:44:59.560606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:59.562100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:59.563627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846103-536d-406b-a438-f0e4c273381e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:44:59.581021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:44:59.582331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:44:59.583146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:44:59.583146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:44:59.583146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:44:59.583146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 9/19] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:44:59.588482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:44:59.625035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:44:59.667927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846103-536d-406b-a438-f0e4c273381e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.669439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:44:59.689510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328dd80-1ba2-45d8-91ab-df28e90be494 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:44:59.702645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8777d-996a-4eb2-abb2-dc79075a7755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.703506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8777d-996a-4eb2-abb2-dc79075a7755 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.703788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8777d-996a-4eb2-abb2-dc79075a7755 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.704057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8777d-996a-4eb2-abb2-dc79075a7755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.704898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8777d-996a-4eb2-abb2-dc79075a7755 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.domain.id=default, user.id=259c064f989f4405a7a84f397fec6223) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:44:59.705076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8777d-996a-4eb2-abb2-dc79075a7755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:59.706539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8777d-996a-4eb2-abb2-dc79075a7755 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:59.715352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 11/20] 10.222.0.72 () {56 vars in 1322 bytes} [Tue Sep 13 12:44:59 2022] GET /identity/v3/role_assignments?user.id=259c064f989f4405a7a84f397fec6223&scope.domain.id=default&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:44:59.726019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216feed9-a336-4a1f-8834-ee85d8f6b52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:44:59.726659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216feed9-a336-4a1f-8834-ee85d8f6b52d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:44:59.726962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216feed9-a336-4a1f-8834-ee85d8f6b52d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:44:59.727947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216feed9-a336-4a1f-8834-ee85d8f6b52d None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:44:59.728644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216feed9-a336-4a1f-8834-ee85d8f6b52d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:44:59.728956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216feed9-a336-4a1f-8834-ee85d8f6b52d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:44:59.730585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216feed9-a336-4a1f-8834-ee85d8f6b52d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:44:59.751140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 10/21] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:02.386548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53f20b-6f54-41da-9133-3785778929ea None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.387064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53f20b-6f54-41da-9133-3785778929ea None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.387174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53f20b-6f54-41da-9133-3785778929ea None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.388713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 12/22] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:02.395158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d60e6c-4549-43cc-9772-797191f6fa83 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.395877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d60e6c-4549-43cc-9772-797191f6fa83 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.395877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d60e6c-4549-43cc-9772-797191f6fa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.418983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-49d60e6c-4549-43cc-9772-797191f6fa83 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:02.420252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-49d60e6c-4549-43cc-9772-797191f6fa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:02.497920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa704a-dc80-4052-99ca-0b6beae757f6 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.498250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa704a-dc80-4052-99ca-0b6beae757f6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.498467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa704a-dc80-4052-99ca-0b6beae757f6 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.499977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 13/23] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:02.505142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d623ce6-4111-4294-85cd-304ec2a110ed None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.505364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d623ce6-4111-4294-85cd-304ec2a110ed None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.505638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d623ce6-4111-4294-85cd-304ec2a110ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.510633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 11/24] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:02.542990 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5d623ce6-4111-4294-85cd-304ec2a110ed None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:02.544260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5d623ce6-4111-4294-85cd-304ec2a110ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:02.624713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da6e9b-6cfc-4639-bab0-efb95e03cb52 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.625041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da6e9b-6cfc-4639-bab0-efb95e03cb52 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.625280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da6e9b-6cfc-4639-bab0-efb95e03cb52 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.626716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 12/25] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:02.640022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 14/26] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 12:45:02.646202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d363f32-861b-4f45-9b6c-519172d5c51c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.646487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d363f32-861b-4f45-9b6c-519172d5c51c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.646931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d363f32-861b-4f45-9b6c-519172d5c51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.686756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4d363f32-861b-4f45-9b6c-519172d5c51c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:02.688028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4d363f32-861b-4f45-9b6c-519172d5c51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:02.715345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:02.749476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb18972-5f6a-496a-941d-832a8ac534b2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.749770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb18972-5f6a-496a-941d-832a8ac534b2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.750224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb18972-5f6a-496a-941d-832a8ac534b2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.751759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 15/27] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:02.757595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e6463-3e1b-4d01-b2b4-6b0deb51bd14 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:02.758120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e6463-3e1b-4d01-b2b4-6b0deb51bd14 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:02.758530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e6463-3e1b-4d01-b2b4-6b0deb51bd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:02.783181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 13/28] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:02.793684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8b9e6463-3e1b-4d01-b2b4-6b0deb51bd14 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:02.795061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8b9e6463-3e1b-4d01-b2b4-6b0deb51bd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:02.890465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 16/29] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:02.971840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d363f32-861b-4f45-9b6c-519172d5c51c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:03.033588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e6463-3e1b-4d01-b2b4-6b0deb51bd14 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:03.088590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d363f32-861b-4f45-9b6c-519172d5c51c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:03.090337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:03.126833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:03.127478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:03.127759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:03.128034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:03.139511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:03.144753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:03.145661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:03.146593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 14/30] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:03.156974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:03.186145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:03.242821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e6463-3e1b-4d01-b2b4-6b0deb51bd14 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:03.291055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:03.292040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:03.292552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:03.292831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:03.295708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2856f8f-8df7-4e6e-ad70-b00d83dc58ef None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:03.325785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:03.326571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:03.345374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:03.346042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:03.346322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:03.346597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:03.347266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:03.347569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:03.348961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:45:03.349979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:03.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:45:03.352760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:45:03.352760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:45:03.352760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:03.354152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 17/31] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:45:03.355114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:03.366310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 15/32] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:03.367209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:03.440373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:03.460672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:03.463235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636ea8-fe24-4cde-8b8f-14be91be06c9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:03.491514 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e949a-f90d-4d87-b0a1-50b264125221 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:03.501614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:03.502542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:03.502885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:03.503245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:03.503982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:03.505428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:03.534749 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:03.535671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:03.536134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:03.536625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:03.537458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:03.539162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:03.720915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 18/33] 10.222.0.72 () {60 vars in 1131 bytes} [Tue Sep 13 12:45:03 2022] POST /identity/v3/roles => generated 219 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:03.725777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:03.725963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 16/34] 10.222.0.72 () {60 vars in 1131 bytes} [Tue Sep 13 12:45:03 2022] POST /identity/v3/roles => generated 225 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:03.725963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:03.969116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b3c20-27d9-4faa-9fa8-0b0a4eb48418 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:03.989015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a2101-b5cc-4a07-9309-d7414fb2f8ea None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:04.053972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:04.055538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.056047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.056552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] PATH_INFO: `/v3/users/259c064f989f4405a7a84f397fec6223` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.058053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] RBAC: Authorizing `identity:get_user(user_id=259c064f989f4405a7a84f397fec6223)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:04.058493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:04.059139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.059433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.059722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.060395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:04.060707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:04.061052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:04.062261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:04.063446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 17/35] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:03 2022] GET /identity/v3/users/259c064f989f4405a7a84f397fec6223 => generated 247 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:04.069829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:04.074185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:04.076389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:04.094544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 19/36] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:04.100029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:04.201717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87238aa-4853-4106-8fc5-24629d5652b5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:04.236651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee470a14-9dca-42c1-8976-e49f096e987a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:04.240222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40b7cf-6ca2-48e2-bf8d-bc0b265ae3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:04.240822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b7cf-6ca2-48e2-bf8d-bc0b265ae3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.241104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b7cf-6ca2-48e2-bf8d-bc0b265ae3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.241379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40b7cf-6ca2-48e2-bf8d-bc0b265ae3f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.242634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40b7cf-6ca2-48e2-bf8d-bc0b265ae3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:04.244078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40b7cf-6ca2-48e2-bf8d-bc0b265ae3f2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:04.250417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 18/37] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:04.250417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ded79-0cff-4bb9-8409-3d9172b9c6b1 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.250417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ded79-0cff-4bb9-8409-3d9172b9c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.250417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ded79-0cff-4bb9-8409-3d9172b9c6b1 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.250417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 19/38] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:04.251318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ded79-0cff-4bb9-8409-3d9172b9c6b1 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:04.279996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cfa64-6892-4d95-acb8-0550af9e7898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:04.280616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cfa64-6892-4d95-acb8-0550af9e7898 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.280900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cfa64-6892-4d95-acb8-0550af9e7898 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.281178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cfa64-6892-4d95-acb8-0550af9e7898 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.281984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cfa64-6892-4d95-acb8-0550af9e7898 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:04.283499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cfa64-6892-4d95-acb8-0550af9e7898 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:04.324087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:04.354989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ded79-0cff-4bb9-8409-3d9172b9c6b1 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:04.379468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 20/39] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:45:04 2022] POST /identity/v3/projects => generated 301 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:45:04.383536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fa4d8-3fee-43ad-904f-69621fe540b9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.383536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fa4d8-3fee-43ad-904f-69621fe540b9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.383536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fa4d8-3fee-43ad-904f-69621fe540b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.389445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5c79a-a53b-408b-b63b-a54f0b2a7e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:04.390168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5c79a-a53b-408b-b63b-a54f0b2a7e02 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.390449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5c79a-a53b-408b-b63b-a54f0b2a7e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.390799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5c79a-a53b-408b-b63b-a54f0b2a7e02 None admin] PATH_INFO: `/v3/domains/default/users/259c064f989f4405a7a84f397fec6223/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.392763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5c79a-a53b-408b-b63b-a54f0b2a7e02 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=259c064f989f4405a7a84f397fec6223, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:04.394303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5c79a-a53b-408b-b63b-a54f0b2a7e02 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:04.412674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-237fa4d8-3fee-43ad-904f-69621fe540b9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:04.415092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-237fa4d8-3fee-43ad-904f-69621fe540b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:04.468867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:04.477083 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 20/40] 10.222.0.72 () {58 vars in 1361 bytes} [Tue Sep 13 12:45:04 2022] PUT /identity/v3/domains/default/users/259c064f989f4405a7a84f397fec6223/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 12:45:04.486022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae996d5-c90f-444a-9ede-b9ee28c92f2f None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.486022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae996d5-c90f-444a-9ede-b9ee28c92f2f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.486022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae996d5-c90f-444a-9ede-b9ee28c92f2f None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.486022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 21/41] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:04.494212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f62bae-b225-4993-9f2a-73da58d6967b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.494452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f62bae-b225-4993-9f2a-73da58d6967b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.494819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f62bae-b225-4993-9f2a-73da58d6967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.499017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 21/42] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:04.632623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-10f62bae-b225-4993-9f2a-73da58d6967b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:04.632623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-10f62bae-b225-4993-9f2a-73da58d6967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:04.715038 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 22/43] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:04.778000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fa4d8-3fee-43ad-904f-69621fe540b9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:04.879043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fa4d8-3fee-43ad-904f-69621fe540b9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:04.914250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:04.915208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:04.915693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:04.916172 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:04.917504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:04.919286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:04.921060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 22/44] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:04.926803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f62bae-b225-4993-9f2a-73da58d6967b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:04.989278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:04.989487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:05.033145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f62bae-b225-4993-9f2a-73da58d6967b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:05.044936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:05.072885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58630d3a-5ceb-4a5d-bd5e-c2b1c9a7f050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:05.073503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58630d3a-5ceb-4a5d-bd5e-c2b1c9a7f050 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:05.073808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58630d3a-5ceb-4a5d-bd5e-c2b1c9a7f050 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:05.074084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58630d3a-5ceb-4a5d-bd5e-c2b1c9a7f050 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:05.074901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58630d3a-5ceb-4a5d-bd5e-c2b1c9a7f050 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:05.076347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58630d3a-5ceb-4a5d-bd5e-c2b1c9a7f050 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:05.100266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7898b-f0a7-451f-9dde-e7e9685a73f9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:05.133954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9faa8-788f-48fc-a5af-d572e5f147b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:05.134558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9faa8-788f-48fc-a5af-d572e5f147b4 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:05.134843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9faa8-788f-48fc-a5af-d572e5f147b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:05.135121 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9faa8-788f-48fc-a5af-d572e5f147b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:05.135801 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9faa8-788f-48fc-a5af-d572e5f147b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:05.137169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9faa8-788f-48fc-a5af-d572e5f147b4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:05.162648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 23/45] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:45:04 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:05.237083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 23/46] 10.222.0.72 () {60 vars in 1131 bytes} [Tue Sep 13 12:45:04 2022] POST /identity/v3/roles => generated 223 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:45:06.676196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d037be-417b-40da-8d25-ab8a6c0083d5 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:06.676667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d037be-417b-40da-8d25-ab8a6c0083d5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:06.676747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d037be-417b-40da-8d25-ab8a6c0083d5 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:06.678104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 24/47] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:06.683358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaf3b3-829a-4a9e-946d-6bf876166b58 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:06.683758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaf3b3-829a-4a9e-946d-6bf876166b58 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:06.683906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaf3b3-829a-4a9e-946d-6bf876166b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:06.701532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9bfaf3b3-829a-4a9e-946d-6bf876166b58 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:06.702988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9bfaf3b3-829a-4a9e-946d-6bf876166b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:06.767795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 24/48] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:07.035295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d037be-417b-40da-8d25-ab8a6c0083d5 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:07.135271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d037be-417b-40da-8d25-ab8a6c0083d5 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:07.149414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:07.168604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318e0b1-55ca-45a2-8c56-1c975f797010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:07.169239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e0b1-55ca-45a2-8c56-1c975f797010 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.169518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e0b1-55ca-45a2-8c56-1c975f797010 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.169826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e0b1-55ca-45a2-8c56-1c975f797010 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.180006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318e0b1-55ca-45a2-8c56-1c975f797010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-e318e0b1-55ca-45a2-8c56-1c975f797010 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:07.186097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:07.186940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:07.187698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 25/49] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:07.193144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaf3b3-829a-4a9e-946d-6bf876166b58 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:07.234863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:07.295853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaf3b3-829a-4a9e-946d-6bf876166b58 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:07.339024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:07.339650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.339936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.340216 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.340892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:07.341201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:07.342638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:07.349592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d2bd-96f9-40f2-b78b-dbcc0826ca44 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.349922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d2bd-96f9-40f2-b78b-dbcc0826ca44 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.350204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6d2bd-96f9-40f2-b78b-dbcc0826ca44 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.351578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 26/50] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:07.352788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 25/51] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:07.358304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:07.358427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3112-546d-4916-8c46-bc73c7eea0c1 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.358709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3112-546d-4916-8c46-bc73c7eea0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.358988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3112-546d-4916-8c46-bc73c7eea0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.377644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-69ff3112-546d-4916-8c46-bc73c7eea0c1 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:07.378902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-69ff3112-546d-4916-8c46-bc73c7eea0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:07.444281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 27/52] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:07.454448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:07.457320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb4cdf-be4d-49b7-a718-37951fdd32a4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:07.489484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:07.490419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.490836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.491462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] PATH_INFO: `/v3/users/259c064f989f4405a7a84f397fec6223` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.493239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] RBAC: Authorizing `identity:get_user(user_id=259c064f989f4405a7a84f397fec6223)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:07.494703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:07.496324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 26/53] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:07 2022] GET /identity/v3/users/259c064f989f4405a7a84f397fec6223 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:07.501863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff3112-546d-4916-8c46-bc73c7eea0c1 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:07.599952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff3112-546d-4916-8c46-bc73c7eea0c1 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:07.602391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:07.634792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:07.635719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.635801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.636024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.637183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:07.638675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:07.640333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 28/54] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:07.646206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:07.716256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:07.746779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680cc50-5763-465b-94b4-97007864a47f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:07.828842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5b110-8243-49b8-8196-0a157477cdcf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:07.865133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:07.865686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.865987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.866260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.868952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:07.870511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:07.872270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 29/55] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:07.877518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:07.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326c6fb-6cdb-4ae1-aed7-bbf2692a1e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:07.888228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c6fb-6cdb-4ae1-aed7-bbf2692a1e20 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:07.888515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c6fb-6cdb-4ae1-aed7-bbf2692a1e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:07.888800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c6fb-6cdb-4ae1-aed7-bbf2692a1e20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:07.889532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326c6fb-6cdb-4ae1-aed7-bbf2692a1e20 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.domain.id=default, user.id=259c064f989f4405a7a84f397fec6223) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:07.889881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326c6fb-6cdb-4ae1-aed7-bbf2692a1e20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:07.891339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326c6fb-6cdb-4ae1-aed7-bbf2692a1e20 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:07.897217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 27/56] 10.222.0.72 () {56 vars in 1322 bytes} [Tue Sep 13 12:45:07 2022] GET /identity/v3/role_assignments?user.id=259c064f989f4405a7a84f397fec6223&scope.domain.id=default&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 530 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:07.921403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:07.982418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adca12c-09ff-43c5-ae67-511eaace5121 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:08.018969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be642c-e552-4ad7-a97e-379595b6a1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:08.019594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be642c-e552-4ad7-a97e-379595b6a1a0 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:08.019877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be642c-e552-4ad7-a97e-379595b6a1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:08.020151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be642c-e552-4ad7-a97e-379595b6a1a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:08.020823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be642c-e552-4ad7-a97e-379595b6a1a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:08.022345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be642c-e552-4ad7-a97e-379595b6a1a0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:08.109198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 30/57] 10.222.0.72 () {60 vars in 1140 bytes} [Tue Sep 13 12:45:07 2022] POST /identity/v3/projects => generated 298 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:45:10.266047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ead9-5bfa-437f-b78d-f2457c953856 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:10.266562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ead9-5bfa-437f-b78d-f2457c953856 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:10.266667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341ead9-5bfa-437f-b78d-f2457c953856 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:10.267943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 28/58] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:10.273248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0930f-4616-47ba-a301-1c4d9d574974 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:10.273764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0930f-4616-47ba-a301-1c4d9d574974 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:10.274074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0930f-4616-47ba-a301-1c4d9d574974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:10.292319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-c7f0930f-4616-47ba-a301-1c4d9d574974 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:10.293519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c7f0930f-4616-47ba-a301-1c4d9d574974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:10.358171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 31/59] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:10.628385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ead9-5bfa-437f-b78d-f2457c953856 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:10.659667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:10.727645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341ead9-5bfa-437f-b78d-f2457c953856 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:10.764797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64743bad-775d-4b7a-a380-636eb80581a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:10.765417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64743bad-775d-4b7a-a380-636eb80581a1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:10.765696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64743bad-775d-4b7a-a380-636eb80581a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:10.766007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64743bad-775d-4b7a-a380-636eb80581a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:10.767115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64743bad-775d-4b7a-a380-636eb80581a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:10.768537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64743bad-775d-4b7a-a380-636eb80581a1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:10.770103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 29/60] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:10.776133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0930f-4616-47ba-a301-1c4d9d574974 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:10.799319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:10.873621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0930f-4616-47ba-a301-1c4d9d574974 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:10.906738 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ee19a-b5ad-492c-9c34-4298a670d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:10.907288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee19a-b5ad-492c-9c34-4298a670d70e None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:10.907568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee19a-b5ad-492c-9c34-4298a670d70e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:10.907842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee19a-b5ad-492c-9c34-4298a670d70e None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:10.908501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ee19a-b5ad-492c-9c34-4298a670d70e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:10.910000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ee19a-b5ad-492c-9c34-4298a670d70e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:11.002683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 32/61] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:45:10 2022] POST /identity/v3/users => generated 275 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:14.476702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef311a-e2a0-4397-8221-f50fcbb8db96 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.477833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef311a-e2a0-4397-8221-f50fcbb8db96 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.478255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef311a-e2a0-4397-8221-f50fcbb8db96 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.479837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 30/62] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:14.485759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caea2b7-865b-442f-8466-01f58a0dbe79 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.486668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caea2b7-865b-442f-8466-01f58a0dbe79 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.487084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caea2b7-865b-442f-8466-01f58a0dbe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.509729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2caea2b7-865b-442f-8466-01f58a0dbe79 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:14.511160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2caea2b7-865b-442f-8466-01f58a0dbe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:14.574994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e013e-25db-4201-8850-25b6e7505f7c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.575537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e013e-25db-4201-8850-25b6e7505f7c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.575959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e013e-25db-4201-8850-25b6e7505f7c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.577814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 31/63] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:14.583803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932240ac-0ef2-4381-9ae7-fcc207ad1c2d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.584110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932240ac-0ef2-4381-9ae7-fcc207ad1c2d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.584345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932240ac-0ef2-4381-9ae7-fcc207ad1c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.607649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-932240ac-0ef2-4381-9ae7-fcc207ad1c2d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:14.609709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-932240ac-0ef2-4381-9ae7-fcc207ad1c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:14.613881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 33/64] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:14.615235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868a9d-4ee7-44f9-9c95-e5c053d22142 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.615529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868a9d-4ee7-44f9-9c95-e5c053d22142 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.615805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868a9d-4ee7-44f9-9c95-e5c053d22142 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.626051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 34/65] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:14.631660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e869c-bcdf-4ff5-bb36-56ef052845cf None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.632269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e869c-bcdf-4ff5-bb36-56ef052845cf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.632382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e869c-bcdf-4ff5-bb36-56ef052845cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.668813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-b60e869c-bcdf-4ff5-bb36-56ef052845cf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:14.670060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b60e869c-bcdf-4ff5-bb36-56ef052845cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:14.676738 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:14.709195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 32/66] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:14.710495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4822-3431-4086-a4c2-b87216fce6b0 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.711456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4822-3431-4086-a4c2-b87216fce6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.711600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4822-3431-4086-a4c2-b87216fce6b0 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.714146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 33/67] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:14.724010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926afa-7f2e-43aa-a2f1-de784b339fd7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.724299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926afa-7f2e-43aa-a2f1-de784b339fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.724577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926afa-7f2e-43aa-a2f1-de784b339fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.744060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a3926afa-7f2e-43aa-a2f1-de784b339fd7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:14.745964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a3926afa-7f2e-43aa-a2f1-de784b339fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:14.774114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:14.809158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 35/68] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:14.819119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 34/69] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:14.966530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e869c-bcdf-4ff5-bb36-56ef052845cf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:14.986346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928912-3664-4a70-affa-4176668c2556 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.986964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928912-3664-4a70-affa-4176668c2556 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.987458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928912-3664-4a70-affa-4176668c2556 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:14.989185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 35/70] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:14.994369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb22eea-6134-42e4-9079-5141fb959d92 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:14.994886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb22eea-6134-42e4-9079-5141fb959d92 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:14.995366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb22eea-6134-42e4-9079-5141fb959d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.015369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3bb22eea-6134-42e4-9079-5141fb959d92 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:15.017385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3bb22eea-6134-42e4-9079-5141fb959d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:15.086995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e869c-bcdf-4ff5-bb36-56ef052845cf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.097722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 36/71] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:15.098828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb22eea-6134-42e4-9079-5141fb959d92 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.150433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.151073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.151350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.151620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.161470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.166711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:15.172182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:15.177884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 36/72] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:15.178637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.221429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.232166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb22eea-6134-42e4-9079-5141fb959d92 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.268517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.269069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.269353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.269635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.281481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:15.283587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fda0af-f760-4f60-9f48-78983d569c10 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:15.288672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:15.289503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:15.290296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 37/73] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:15.290915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.324105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.324745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.325031 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.325312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.336426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:15.341352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:15.342244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:15.343010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 37/74] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:15.344144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.402881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e3b4-76ed-436c-a4a7-eaca519a58c7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.406719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.439092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.439706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.440043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.440328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.441000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:15.441307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:15.442722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:15.449628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d426836-d82e-4fd3-bb83-c1cae7079ca6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.451354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 38/75] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:15.452164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.470342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.483775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.484414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.484695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.486614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.496527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:15.501681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:15.502550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:15.503329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 38/76] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:15.503966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.556763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5526c1-b453-49d1-99c8-b198ad5bbfb3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.590589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.591194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.591478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.591757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.592423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:15.592726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:15.594126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:15.602389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 39/77] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:15.603173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.603865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20346b-17bf-4b75-a777-1016919db80b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.623453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.641600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.642242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.642528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.642799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.643476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:15.643782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:15.645181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:15.653567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 39/78] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:15.654213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.680168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.704323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45850d4b-aaf7-49bf-bd4c-8680ead13a91 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.737635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.738311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.738599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.738879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.740006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:15.741454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:15.743631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 40/79] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:15.748193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.755063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1130a-7bda-441b-9a01-8d1053e7f668 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.788438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.789194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.789559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.789875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.802927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:15.804896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:15.806647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 40/80] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:15.807467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.843205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c3f7d-d682-42a0-95e3-d085d8098175 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.849807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.876483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.877084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.877366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.877637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.878326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:15.878635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:15.880005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:15.889069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 41/81] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:15.889813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.893323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:15.910841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81844-87f2-4b56-807e-e804c4a89a1c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:15.944093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:15.944730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:15.945645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:15.945950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:15.947167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:15.948622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:15.950249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 41/82] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:15.951011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:15.992589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b6ffb-8eb7-48d9-bf72-dabde54d40e4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.028241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.028861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.029139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.029413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:16.030581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:16.031997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:16.033719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 42/83] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:15 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:16.034403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:16.049311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff13ffc-f67e-4e7e-9039-a32501eab91b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.056577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.083214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.083775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.084055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.084335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:16.084978 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:16.087048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:16.105821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.134250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e3580-8a1e-41ff-8a1f-d60c6ad2a2c3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.166585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.167182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.167469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.167844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:16.174081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 42/84] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:45:15 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:16.174081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:16.245973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:16.248438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:16.317251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c174b-d386-48cc-b66e-1589efc0ac27 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.334597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.356664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.357351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.357585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.357893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] PATH_INFO: `/v3/users/259c064f989f4405a7a84f397fec6223` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:16.359045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] RBAC: Authorizing `identity:get_user(user_id=259c064f989f4405a7a84f397fec6223)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:16.360494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:16.362256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 43/85] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:16 2022] GET /identity/v3/users/259c064f989f4405a7a84f397fec6223 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:16.362968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:16.375794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 43/86] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:16.377529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:16.399745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.480389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b56cf-304f-487a-a937-ec76f9ca300e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.506003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da2e09-4ad1-4c7a-b47d-8174d08b3e79 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.529168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.530477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.530772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.531060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:16.549373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.550030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.550425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.550707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:16.606728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:16.608006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.608431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:16.642647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:16.645228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:16.650372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.743300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 44/87] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:16.744495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:16.765048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 44/88] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:16.770132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:16.861830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.884124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc14854-8e01-4db9-927d-f6e0ab4cc82c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.899463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d6649-ea8a-449b-8a98-9a6cad1652db None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:16.935561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.936200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.936487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.936767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:16.937925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:16.939365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:16.941160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 45/89] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:16 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:16.941782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:16.965252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:16.974339 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:16.975233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:16.975544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:16.979368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.052107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.053959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.091719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da751d-8545-4449-8497-c4e87b21e038 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:17.137118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:17.137675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:17.138003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:17.138283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.139689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.141344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.142949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 46/90] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:16 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:17.143473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:17.157098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:17.193623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 45/91] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:17.194962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:17.233103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:17.280092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327af071-52e9-4e00-b1d4-764139d760a8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:17.312360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a90959-24c7-4e3c-8169-b734b4fdf41b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:17.318579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:17.319215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:17.319497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:17.319772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] PATH_INFO: `/v3/projects/f8f79cdf214b4d86b6f1572f6343002f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.321835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=f8f79cdf214b4d86b6f1572f6343002f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.323265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.325010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 47/92] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:17 2022] GET /identity/v3/projects/f8f79cdf214b4d86b6f1572f6343002f => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:17.325716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:17.355460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:17.356461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:17.356942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:17.357412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.358421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=d332b1b14aa24ef8b6ae8d8929faf9ca) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:17.358935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.360657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.369022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 46/93] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:17 2022] GET /identity/v3/role_assignments?user.id=d332b1b14aa24ef8b6ae8d8929faf9ca&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:17.370151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:17.460054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:17.514588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe164b-f038-44fe-b228-201848d67bd2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:17.577619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:17.589043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce630a3-6c97-4cd7-a19d-ed2cfd186104 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:17.599897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:17.601642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:17.602371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:17.603120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.604333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] RBAC: Adding query filter params (role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=d332b1b14aa24ef8b6ae8d8929faf9ca) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:17.604818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.606515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.618287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 48/94] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:17 2022] GET /identity/v3/role_assignments?user.id=d332b1b14aa24ef8b6ae8d8929faf9ca&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:17.618287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:17.656747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:17.658142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:17.658735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:17.659152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.660655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.662503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.669014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 47/95] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:17 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:17.672156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:17.796272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da90176-0a70-46c3-9a75-e6012f8f9aae None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:17.843921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b11ec-3f1b-4cfa-b576-7c6901afb699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:17.844786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b11ec-3f1b-4cfa-b576-7c6901afb699 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:17.845207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b11ec-3f1b-4cfa-b576-7c6901afb699 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:17.845643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b11ec-3f1b-4cfa-b576-7c6901afb699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.846714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b11ec-3f1b-4cfa-b576-7c6901afb699 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=259c064f989f4405a7a84f397fec6223) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:17.847168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b11ec-3f1b-4cfa-b576-7c6901afb699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.849045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b11ec-3f1b-4cfa-b576-7c6901afb699 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.862106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 48/96] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:17 2022] GET /identity/v3/role_assignments?user.id=259c064f989f4405a7a84f397fec6223&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:17.874664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7f0c-5f7e-4226-9cac-1ecbbc70fc11 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:17.942643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6525dbd-4662-489f-a52e-3b4297cc120d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:17.943357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6525dbd-4662-489f-a52e-3b4297cc120d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:17.943649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6525dbd-4662-489f-a52e-3b4297cc120d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:17.943929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6525dbd-4662-489f-a52e-3b4297cc120d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:17.944685 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6525dbd-4662-489f-a52e-3b4297cc120d None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=f8f79cdf214b4d86b6f1572f6343002f, user.id=d332b1b14aa24ef8b6ae8d8929faf9ca) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:17.945704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6525dbd-4662-489f-a52e-3b4297cc120d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:17.947261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6525dbd-4662-489f-a52e-3b4297cc120d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:17.954088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 49/97] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:17 2022] GET /identity/v3/role_assignments?user.id=d332b1b14aa24ef8b6ae8d8929faf9ca&scope.project.id=f8f79cdf214b4d86b6f1572f6343002f&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:18.693922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e47367-b86b-4705-9185-1ec25339975d None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:18.694949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e47367-b86b-4705-9185-1ec25339975d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:18.695678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e47367-b86b-4705-9185-1ec25339975d None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:18.697310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 49/98] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:18.703326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d86fb-8a4c-42b0-88a7-6c0f38becff1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:18.704193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d86fb-8a4c-42b0-88a7-6c0f38becff1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:18.704608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d86fb-8a4c-42b0-88a7-6c0f38becff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:18.715269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:18.730689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d05d86fb-8a4c-42b0-88a7-6c0f38becff1 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:18.732121 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d05d86fb-8a4c-42b0-88a7-6c0f38becff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:18.770831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf8d44-69aa-4a20-9620-7f9ca7e2e366 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:18.771281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf8d44-69aa-4a20-9620-7f9ca7e2e366 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:18.771657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf8d44-69aa-4a20-9620-7f9ca7e2e366 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:18.773303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 50/99] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:18.778860 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957e432-c205-42b5-bd16-10141d6e0c06 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:18.779263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957e432-c205-42b5-bd16-10141d6e0c06 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:18.779646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957e432-c205-42b5-bd16-10141d6e0c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:18.814778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6957e432-c205-42b5-bd16-10141d6e0c06 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:18.816189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6957e432-c205-42b5-bd16-10141d6e0c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:18.828044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 50/100] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:18.853064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:18.972049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 51/101] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:19.366047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d86fb-8a4c-42b0-88a7-6c0f38becff1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:19.440803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957e432-c205-42b5-bd16-10141d6e0c06 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:19.520592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d86fb-8a4c-42b0-88a7-6c0f38becff1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:19.580957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:19.581987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:19.582491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:19.582951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:19.586490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:19.587967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:19.589570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 51/102] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:19.600349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:19.603848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:19.630057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957e432-c205-42b5-bd16-10141d6e0c06 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:19.727297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:19.727937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:19.728226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:19.728506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:19.763464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:19.768391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:19.769243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:19.770017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:19.774091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 52/103] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:19.780242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:19.792093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06226cd4-8d7a-4272-a91d-77e902770e49 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:19.829122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:19.871530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:19.872955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:19.873245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:19.873520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:19.874213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:19.875646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:19.975826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 52/104] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:45:19 2022] POST /identity/v3/projects => generated 302 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:20.000698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e8e9-bb84-4188-b279-ed6a24fb0a42 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:20.056297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:20.056927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:20.057387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:20.057671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:20.058388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:20.058698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:20.060102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:20.074171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 53/105] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:20.080124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:20.162963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:20.211248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87538723-d695-4c8a-a970-a44f1cc11521 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:20.247367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:20.248020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:20.248227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:20.248914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] PATH_INFO: `/v3/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:20.267421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] RBAC: Authorizing `identity:get_user(user_id=cdaf633bc1d0459aaa2b9f2e4ba9edf0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:20.268882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:20.270628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 53/106] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:20 2022] GET /identity/v3/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:20.276569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:20.375061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:20.399655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194814c-19bc-4f2d-8ff0-3388f28bae9e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:20.446407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659dace-c4dc-4d83-91f1-2723a6331d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:20.447032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659dace-c4dc-4d83-91f1-2723a6331d8a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:20.447320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659dace-c4dc-4d83-91f1-2723a6331d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:20.447597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659dace-c4dc-4d83-91f1-2723a6331d8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:20.541867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659dace-c4dc-4d83-91f1-2723a6331d8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:20.543364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659dace-c4dc-4d83-91f1-2723a6331d8a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:20.650164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:20.674353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 54/107] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:20.680275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:20.824609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08ba0b-8e4c-417f-9dcd-8f836884f556 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:20.829655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e4569-3b28-414a-8ef1-b046f5e0b48e None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:20.830103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e4569-3b28-414a-8ef1-b046f5e0b48e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:20.830269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e4569-3b28-414a-8ef1-b046f5e0b48e None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:20.831608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 55/108] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:20.839896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5312c-bed0-46c6-82d5-27e69c297c25 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:20.840553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5312c-bed0-46c6-82d5-27e69c297c25 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:20.840846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5312c-bed0-46c6-82d5-27e69c297c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:20.855577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:20.884691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-48f5312c-bed0-46c6-82d5-27e69c297c25 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:20.885994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-48f5312c-bed0-46c6-82d5-27e69c297c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:20.914790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:20.915470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:20.915760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:20.916044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:20.917273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:20.918834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:20.920862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 54/109] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:20 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:20.926529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:21.062736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 56/110] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:21.083109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23537a-fe41-4ff4-b22b-8f52b6f74846 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.083368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23537a-fe41-4ff4-b22b-8f52b6f74846 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.083597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23537a-fe41-4ff4-b22b-8f52b6f74846 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.085065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 57/111] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:21.086042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf01984-5629-4007-9cc7-25d95d0113ce None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:21.090615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26726-85c7-4d84-848c-e4f8331c8d08 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.090906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26726-85c7-4d84-848c-e4f8331c8d08 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.091187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed26726-85c7-4d84-848c-e4f8331c8d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.117441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7ed26726-85c7-4d84-848c-e4f8331c8d08 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:21.118687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7ed26726-85c7-4d84-848c-e4f8331c8d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:21.162647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1ee63-c24c-43e7-aa19-ade390bf9b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:21.163266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ee63-c24c-43e7-aa19-ade390bf9b4b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.164234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ee63-c24c-43e7-aa19-ade390bf9b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.164516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1ee63-c24c-43e7-aa19-ade390bf9b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.165275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1ee63-c24c-43e7-aa19-ade390bf9b4b None admin] RBAC: Adding query filter params (role.id=8f099e1c603944dfb66a49bef8b0b790, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=cdaf633bc1d0459aaa2b9f2e4ba9edf0) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:21.165577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1ee63-c24c-43e7-aa19-ade390bf9b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:21.167658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1ee63-c24c-43e7-aa19-ade390bf9b4b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:21.176514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 55/112] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:20 2022] GET /identity/v3/role_assignments?user.id=cdaf633bc1d0459aaa2b9f2e4ba9edf0&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=8f099e1c603944dfb66a49bef8b0b790 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:21.176514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5251ee-c6f1-4a20-8c9a-dcf2ca7e4a1f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.176514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5251ee-c6f1-4a20-8c9a-dcf2ca7e4a1f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.176514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5251ee-c6f1-4a20-8c9a-dcf2ca7e4a1f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.176514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 56/113] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:21.183250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a48a2-8ab1-4b63-8aa4-483e50ecb9b9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.183893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a48a2-8ab1-4b63-8aa4-483e50ecb9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.185688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a48a2-8ab1-4b63-8aa4-483e50ecb9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.230485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a48a48a2-8ab1-4b63-8aa4-483e50ecb9b9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:21.232058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a48a48a2-8ab1-4b63-8aa4-483e50ecb9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:21.279391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 58/114] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:21.311411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:21.356538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 57/115] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:21.419129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed26726-85c7-4d84-848c-e4f8331c8d08 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:21.470909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:21.491379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc01c7-f4cb-4bfc-825a-81932f35a2f1 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.492334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc01c7-f4cb-4bfc-825a-81932f35a2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.492754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc01c7-f4cb-4bfc-825a-81932f35a2f1 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.494375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 58/116] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:21.499898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe04be-50fc-47b9-8746-68b21d361011 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.500183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe04be-50fc-47b9-8746-68b21d361011 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.500484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe04be-50fc-47b9-8746-68b21d361011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.521910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bdbe04be-50fc-47b9-8746-68b21d361011 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:21.523267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bdbe04be-50fc-47b9-8746-68b21d361011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:21.565885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed26726-85c7-4d84-848c-e4f8331c8d08 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:21.619352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:21.619919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.620196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.620464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.633641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 59/117] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:21.644714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:21.648651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:21.649591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:21.650377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 59/118] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:21.655653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe04be-50fc-47b9-8746-68b21d361011 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:21.774366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:21.806503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe04be-50fc-47b9-8746-68b21d361011 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:21.812563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:21.875634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:21.877023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.877535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.878033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.878896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:21.879386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:21.880945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:21.901596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 60/119] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:21.903769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:21.926272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:21.934559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a86b52-4362-4cf1-83bd-465178427a2d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:21.984740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:21.985409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:21.985693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:21.986002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:21.999123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:22.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:22.005602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:22.006376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 60/120] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:45:22.007012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.096971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198afe10-b377-44ff-9967-f6d0601f7dc0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.135506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833afcc-81e7-46bf-992e-6243517a5968 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.143129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.143771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.144057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.144353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.158497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:22.165645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:22.166624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:22.167380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 61/121] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:22.167989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.184057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.185196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.185655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.186116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.188155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.190433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.192542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 61/122] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:22.194012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.205227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:22.230309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:22.300438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66067a-f055-4d6d-930b-9e01c4c72859 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.317353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf1657-d41d-40d3-9dc7-8f57dbf9ed24 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.343941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.344868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.345286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.345706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.347184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:22.347876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.349601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.355342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.355720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.356007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.356286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.359622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 62/123] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:22.364205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.375932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:22.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:22.385495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:22.386342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:22.389077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 62/124] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:22.390300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.467438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:22.481206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acbd34-4a77-4c27-8b1d-d9517b070bb2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.490442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:22.508791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7034ea-fe57-46a6-accc-36ae2855b47c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.527014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.527857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.528303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.528712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.529514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:22.529978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.532359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.543435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 63/125] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:22.544452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.551844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.552644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.553060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.553506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.555877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.557501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.559301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 63/126] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:22.564942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.666798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5618aa-25c6-4339-9be2-1d0a492155bc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.706053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7698b-7f69-465b-aefd-b43d2d8ed16f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.712635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.713454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.714426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.714857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.716661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.718718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.720808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 64/127] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:22.727070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.744072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.744828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.745288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.745691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.746522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:22.747119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.748452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.751151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:22.757164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 64/128] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:22.758632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.778469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:22.849086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bef2f9-10ba-42ee-8fcf-0fc0fd2032b1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.865610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d8902-b4a1-4a0c-9df8-cb5170afde3a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:22.892820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.894751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.895207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.895860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] PATH_INFO: `/v3/users/259c064f989f4405a7a84f397fec6223` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.897247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=259c064f989f4405a7a84f397fec6223)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.899201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.901272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 65/129] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/users/259c064f989f4405a7a84f397fec6223 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:22.902214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:22.907168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:22.908194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:22.908579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:22.908956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/d332b1b14aa24ef8b6ae8d8929faf9ca/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:22.911037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] RBAC: Authorizing `identity:create_grant(project_id=7a1d4145c5484035a6842313c2648cc1, user_id=d332b1b14aa24ef8b6ae8d8929faf9ca, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:22.912587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:22.986834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 65/130] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:22 2022] PUT /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/d332b1b14aa24ef8b6ae8d8929faf9ca/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:22.992643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:23.013941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:23.045840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5985e50-8c11-4291-8bab-a403a69e0bb0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:23.110137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:23.111065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.111395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.111738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] PATH_INFO: `/v3/projects/f8f79cdf214b4d86b6f1572f6343002f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.113718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] RBAC: Authorizing `identity:get_project(project_id=f8f79cdf214b4d86b6f1572f6343002f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:23.115498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:23.118910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 66/131] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/projects/f8f79cdf214b4d86b6f1572f6343002f => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:23.119617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:23.137460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:23.163257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1b689-55aa-430e-adc2-f9d7ff57b15a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:23.210113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6b125-6f87-4b09-99ed-ea0c4c7094dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:23.211669 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6b125-6f87-4b09-99ed-ea0c4c7094dc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.212096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6b125-6f87-4b09-99ed-ea0c4c7094dc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.212665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6b125-6f87-4b09-99ed-ea0c4c7094dc None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.214657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6b125-6f87-4b09-99ed-ea0c4c7094dc None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:23.216425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6b125-6f87-4b09-99ed-ea0c4c7094dc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:23.218852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 66/132] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:22 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:23.220376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dcd73-309b-400e-b88f-40debb9914e0 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.220662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dcd73-309b-400e-b88f-40debb9914e0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.220936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dcd73-309b-400e-b88f-40debb9914e0 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.222807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 67/133] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:23.222912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dcd73-309b-400e-b88f-40debb9914e0 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:23.273646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729b4e-d2f8-4651-affc-97270b47ce39 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:23.310242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:23.310889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.311174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.311455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.312607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:23.314085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:23.315734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 67/134] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:23 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:23.316509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:23.347783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:23.352322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dcd73-309b-400e-b88f-40debb9914e0 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:23.388555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:23.389295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.389586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.389895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] PATH_INFO: `/v3/projects/f8f79cdf214b4d86b6f1572f6343002f/users/d332b1b14aa24ef8b6ae8d8929faf9ca/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.391825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] RBAC: Authorizing `identity:create_grant(project_id=f8f79cdf214b4d86b6f1572f6343002f, user_id=d332b1b14aa24ef8b6ae8d8929faf9ca, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:23.393276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:23.442414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79258ea3-65f5-45ea-b0e2-64d1d2fa8553 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:23.481090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677e103-e025-47d4-bdc3-cbcefffc4240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:23.481760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677e103-e025-47d4-bdc3-cbcefffc4240 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.482046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677e103-e025-47d4-bdc3-cbcefffc4240 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.482327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677e103-e025-47d4-bdc3-cbcefffc4240 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.483447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677e103-e025-47d4-bdc3-cbcefffc4240 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:23.484895 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677e103-e025-47d4-bdc3-cbcefffc4240 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:23.491447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 68/135] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:23 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:23.491447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db746-a271-4e5b-bb90-dfb5b003ae13 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.491447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db746-a271-4e5b-bb90-dfb5b003ae13 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.491447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db746-a271-4e5b-bb90-dfb5b003ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.510777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 68/136] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:23 2022] PUT /identity/v3/projects/f8f79cdf214b4d86b6f1572f6343002f/users/d332b1b14aa24ef8b6ae8d8929faf9ca/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:23.511617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:23.516526 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a01db746-a271-4e5b-bb90-dfb5b003ae13 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:23.518393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a01db746-a271-4e5b-bb90-dfb5b003ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:23.532541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:23.616889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 69/137] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:23.617497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01db746-a271-4e5b-bb90-dfb5b003ae13 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:23.670776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:23.748647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e20390-1f2b-4dad-b7b7-b9247414c22f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:23.816261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01db746-a271-4e5b-bb90-dfb5b003ae13 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:23.827044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1770f-52da-4863-91d9-14fb904057ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:23.827979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1770f-52da-4863-91d9-14fb904057ca None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.828266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1770f-52da-4863-91d9-14fb904057ca None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.828615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1770f-52da-4863-91d9-14fb904057ca None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/259c064f989f4405a7a84f397fec6223/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.831571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1770f-52da-4863-91d9-14fb904057ca None admin] RBAC: Authorizing `identity:create_grant(project_id=7a1d4145c5484035a6842313c2648cc1, user_id=259c064f989f4405a7a84f397fec6223, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:23.833073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1770f-52da-4863-91d9-14fb904057ca None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:23.864132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c622781-720b-46e8-a438-736b19de857a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:23.864811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c622781-720b-46e8-a438-736b19de857a None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.865098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c622781-720b-46e8-a438-736b19de857a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.865424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c622781-720b-46e8-a438-736b19de857a None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/d332b1b14aa24ef8b6ae8d8929faf9ca/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.868392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c622781-720b-46e8-a438-736b19de857a None admin] RBAC: Authorizing `identity:create_grant(project_id=7a1d4145c5484035a6842313c2648cc1, user_id=d332b1b14aa24ef8b6ae8d8929faf9ca, role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:23.869955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c622781-720b-46e8-a438-736b19de857a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:23.916635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 69/138] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:23 2022] PUT /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/259c064f989f4405a7a84f397fec6223/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:23.918061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09133aa-7023-41be-853f-bedecac1cd50 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.918405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09133aa-7023-41be-853f-bedecac1cd50 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.918689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09133aa-7023-41be-853f-bedecac1cd50 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.919996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 70/139] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:23.926089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c48e9-3799-4fd6-8d13-661395dc0cb2 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:23.926425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c48e9-3799-4fd6-8d13-661395dc0cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:23.926711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c48e9-3799-4fd6-8d13-661395dc0cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:23.950954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-cc1c48e9-3799-4fd6-8d13-661395dc0cb2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:23.952578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cc1c48e9-3799-4fd6-8d13-661395dc0cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:23.973027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 70/140] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:23 2022] PUT /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/d332b1b14aa24ef8b6ae8d8929faf9ca/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:24.043229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:24.052036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 71/141] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:24.093598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c622781-720b-46e8-a438-736b19de857a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:24.212359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:24.234335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c622781-720b-46e8-a438-736b19de857a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:24.280176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:24.280850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:24.281139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:24.281419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:24.286035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:24.287530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:24.289864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 71/142] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:24.295983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c48e9-3799-4fd6-8d13-661395dc0cb2 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:24.431325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c48e9-3799-4fd6-8d13-661395dc0cb2 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:24.470780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:24.471400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:24.471686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:24.471964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:24.472623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:24.474099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:24.493223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:24.594863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 72/143] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:45:24 2022] POST /identity/v3/users => generated 283 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:24.632754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56905a3d-6cbb-4f84-9d32-7001c098c883 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:24.681589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:24.682192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:24.682479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:24.682756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:24.699728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:24.705046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:24.705985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:24.706721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 72/144] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:45:24.712989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:24.777016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:24.831530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc20411-6749-4eaf-b663-48ab8a188ff9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:24.875262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:24.875908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:24.876191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:24.876467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:24.877173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:24.877553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:24.879100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:24.894480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 73/145] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:24.900968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:24.913370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:25.012785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceefb4f-ab0a-445e-8c32-8c13de748ecf None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:25.049753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:25.050418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:25.050701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:25.050979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] PATH_INFO: `/v3/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:25.052124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] RBAC: Authorizing `identity:get_user(user_id=cdaf633bc1d0459aaa2b9f2e4ba9edf0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:25.053579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:25.056160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 73/146] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:24 2022] GET /identity/v3/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:25.061509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:25.188200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684270b4-b5bc-45fb-9622-d550ced764f2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:25.202485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:25.225650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ecbde-9db1-4fab-a647-262c78b53af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:25.227247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ecbde-9db1-4fab-a647-262c78b53af8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:25.227579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ecbde-9db1-4fab-a647-262c78b53af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:25.227858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ecbde-9db1-4fab-a647-262c78b53af8 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:25.231291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ecbde-9db1-4fab-a647-262c78b53af8 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:25.232775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ecbde-9db1-4fab-a647-262c78b53af8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:25.234473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 74/147] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:25 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:25.243447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:25.339436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:25.371497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b46aa-90db-4bd0-9345-83871f9f1b91 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:25.416341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e34e78-5b1c-438f-87cd-86515cf4db3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:25.417022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e34e78-5b1c-438f-87cd-86515cf4db3e None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:25.417306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e34e78-5b1c-438f-87cd-86515cf4db3e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:25.417578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e34e78-5b1c-438f-87cd-86515cf4db3e None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0/roles/8f099e1c603944dfb66a49bef8b0b790` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:25.419585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e34e78-5b1c-438f-87cd-86515cf4db3e None admin] RBAC: Authorizing `identity:create_grant(project_id=7a1d4145c5484035a6842313c2648cc1, user_id=cdaf633bc1d0459aaa2b9f2e4ba9edf0, role_id=8f099e1c603944dfb66a49bef8b0b790)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:25.421062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e34e78-5b1c-438f-87cd-86515cf4db3e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:25.510380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 74/148] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:25 2022] PUT /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0/roles/8f099e1c603944dfb66a49bef8b0b790 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 12:45:26.125366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631569f-4e39-4e74-a773-fb7dc75dbadd None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:26.126220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631569f-4e39-4e74-a773-fb7dc75dbadd None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:26.126220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631569f-4e39-4e74-a773-fb7dc75dbadd None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:26.127697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 75/149] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:26.143702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a81839-a617-470c-b692-b04ca9c670eb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:26.144192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a81839-a617-470c-b692-b04ca9c670eb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:26.144262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a81839-a617-470c-b692-b04ca9c670eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:26.176006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f4a81839-a617-470c-b692-b04ca9c670eb None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:26.177228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f4a81839-a617-470c-b692-b04ca9c670eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:26.268714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 75/150] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:26.778059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631569f-4e39-4e74-a773-fb7dc75dbadd None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:26.906226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631569f-4e39-4e74-a773-fb7dc75dbadd None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:26.948554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1973e-cb0a-42b2-8e96-3fa133f0314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:26.949199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1973e-cb0a-42b2-8e96-3fa133f0314d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:26.949539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1973e-cb0a-42b2-8e96-3fa133f0314d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:26.949842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1973e-cb0a-42b2-8e96-3fa133f0314d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:26.967104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1973e-cb0a-42b2-8e96-3fa133f0314d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-75f1973e-cb0a-42b2-8e96-3fa133f0314d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:26.972447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:26.973316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:26.974136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 76/151] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:26.979863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a81839-a617-470c-b692-b04ca9c670eb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:26.999691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d162b1-dabd-4b5b-9e03-878671a54487 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:27.000036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d162b1-dabd-4b5b-9e03-878671a54487 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:27.000239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d162b1-dabd-4b5b-9e03-878671a54487 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:27.001906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 77/152] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:27.009019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ac70d-1ac2-4cb5-8cbc-0b27b86b3b20 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:27.009322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ac70d-1ac2-4cb5-8cbc-0b27b86b3b20 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:27.009570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ac70d-1ac2-4cb5-8cbc-0b27b86b3b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:27.050723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9d6ac70d-1ac2-4cb5-8cbc-0b27b86b3b20 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:27.051935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9d6ac70d-1ac2-4cb5-8cbc-0b27b86b3b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:27.125448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:27.142410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:27.214074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a81839-a617-470c-b692-b04ca9c670eb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:27.230264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 78/153] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:27.265309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:27.265959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:27.266243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:27.266522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:27.267207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:27.267512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:27.268896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:27.277879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 76/154] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:27.283523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ac70d-1ac2-4cb5-8cbc-0b27b86b3b20 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:27.408739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ac70d-1ac2-4cb5-8cbc-0b27b86b3b20 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:27.448793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b1ca-e79f-4d46-a022-12c61d35469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:27.449438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b1ca-e79f-4d46-a022-12c61d35469b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:27.449718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b1ca-e79f-4d46-a022-12c61d35469b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:27.450015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b1ca-e79f-4d46-a022-12c61d35469b None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:27.454410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69b1ca-e79f-4d46-a022-12c61d35469b None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:27.455871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69b1ca-e79f-4d46-a022-12c61d35469b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:27.457456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 79/155] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:27 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:27.463782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:27.578317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dfbe1-f49c-406a-abe7-53f8d92d0e74 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:27.583211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dfbe1-f49c-406a-abe7-53f8d92d0e74 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:27.583738 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dfbe1-f49c-406a-abe7-53f8d92d0e74 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:27.586052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 80/156] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:27.586701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40b8d7-05dc-4bbe-9724-d1257af94c56 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:27.603503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5afa3b-2142-4738-8a2c-b97bbcfa4266 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:27.604113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5afa3b-2142-4738-8a2c-b97bbcfa4266 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:27.604584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5afa3b-2142-4738-8a2c-b97bbcfa4266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:27.619475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:27.636712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9c5afa3b-2142-4738-8a2c-b97bbcfa4266 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:27.645784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9c5afa3b-2142-4738-8a2c-b97bbcfa4266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:27.645978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:27.645978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:27.645978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:27.645978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:27.701831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:27.775582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 81/157] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:27.792703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:27.794321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:27.978846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5afa3b-2142-4738-8a2c-b97bbcfa4266 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:27.984612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 77/158] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:27.990464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:28.043124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:28.150175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5afa3b-2142-4738-8a2c-b97bbcfa4266 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:28.187685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260de47-653b-49d3-b646-8ce300038e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:28.188388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260de47-653b-49d3-b646-8ce300038e86 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.188671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260de47-653b-49d3-b646-8ce300038e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.188953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260de47-653b-49d3-b646-8ce300038e86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.200683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260de47-653b-49d3-b646-8ce300038e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-5260de47-653b-49d3-b646-8ce300038e86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:28.205913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:28.206772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:28.207526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 82/159] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:28.209646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf847c0-e8de-4305-a227-fe7ec46fa255 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.209960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf847c0-e8de-4305-a227-fe7ec46fa255 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.211193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf847c0-e8de-4305-a227-fe7ec46fa255 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.215203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 83/160] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:28.217948 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf847c0-e8de-4305-a227-fe7ec46fa255 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:28.246288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce1d6e-cacc-4d86-a466-13c5e8044fca None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:28.266136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:28.326731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b3cdc-9958-4ce3-bfb5-851eba2918c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:28.327334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf847c0-e8de-4305-a227-fe7ec46fa255 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:28.327637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b3cdc-9958-4ce3-bfb5-851eba2918c0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.327928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b3cdc-9958-4ce3-bfb5-851eba2918c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.328205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b3cdc-9958-4ce3-bfb5-851eba2918c0 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.334346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b3cdc-9958-4ce3-bfb5-851eba2918c0 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:28.336554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b3cdc-9958-4ce3-bfb5-851eba2918c0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:28.338370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 78/161] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:27 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:28.339540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89baccd7-480e-497c-bd43-3cc18f89f7af None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.339807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89baccd7-480e-497c-bd43-3cc18f89f7af None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.340083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89baccd7-480e-497c-bd43-3cc18f89f7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.364525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:28.365201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.365457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.365755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.367002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:28.367848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:28.369401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:28.374747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-89baccd7-480e-497c-bd43-3cc18f89f7af None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:28.375975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-89baccd7-480e-497c-bd43-3cc18f89f7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:28.378097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 84/162] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:28.379004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:28.484830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68df2f-6ff7-40e3-8a01-9ebffe96f960 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:28.493299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:28.495044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 79/163] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:28.495918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89baccd7-480e-497c-bd43-3cc18f89f7af None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:28.558215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:28.558540 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:28.559233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.559495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.559865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.587634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:28.593123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:28.594026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:28.594864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:28.598360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 85/164] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:28.599320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:28.674377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89baccd7-480e-497c-bd43-3cc18f89f7af None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:28.728442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:28.729063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.729344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.729617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.730389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=d332b1b14aa24ef8b6ae8d8929faf9ca) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:28.730700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:28.732138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:28.732500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97bcf9-36ca-4309-9ff4-609e89a2465b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:28.747523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 80/165] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:28 2022] GET /identity/v3/role_assignments?user.id=d332b1b14aa24ef8b6ae8d8929faf9ca&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:28.748612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:28.770667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:28.771315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:28.771597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:28.771874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:28.782758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:28.784269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:28.790673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 86/166] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:28 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:28.793546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:28.843941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:28.953368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:28.974998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40feb84e-d40e-402b-801b-d95dec571f2f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.047653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.048671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.048845 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59fbc1-d242-404f-b1ba-6000a5ba94cb None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.049308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.049816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.050826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:29.051325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:29.052910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:29.076270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 81/167] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:29.082475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:29.127252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.127801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.128231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.128505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.231387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f83ad-21dd-4794-8bef-e08d6d81f637 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.269634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.271077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.271547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.272006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] PATH_INFO: `/v3/users/259c064f989f4405a7a84f397fec6223` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.274077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] RBAC: Authorizing `identity:get_user(user_id=259c064f989f4405a7a84f397fec6223)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:29.275862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:29.277822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 82/168] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:29 2022] GET /identity/v3/users/259c064f989f4405a7a84f397fec6223 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:29.278774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:29.289969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:29.291462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:29.309558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:29.312246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:29.395791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e403486-863c-449d-9ad2-b13b52d0a06c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.439009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.440106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.440616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.441093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.450650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 87/169] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:29.452769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:29.458163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:29.463593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:29.464598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:29.465389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:29.467911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 83/170] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:29.469076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:29.571585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:29.596294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075263f-3cfa-450d-9b02-0d1db3cd3ce9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.604852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164f3c0-eb9c-4bdc-8900-635411be5211 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.636246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:29.650755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.650755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.650755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.650755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] PATH_INFO: `/v3/projects/f8f79cdf214b4d86b6f1572f6343002f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.651160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] RBAC: Authorizing `identity:get_project(project_id=f8f79cdf214b4d86b6f1572f6343002f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:29.651570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:29.653393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 84/171] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:29 2022] GET /identity/v3/projects/f8f79cdf214b4d86b6f1572f6343002f => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:29.654454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:29.701946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13490a-50be-4d5b-a9f4-e1541e921dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.702826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13490a-50be-4d5b-a9f4-e1541e921dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.703112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13490a-50be-4d5b-a9f4-e1541e921dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.703394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13490a-50be-4d5b-a9f4-e1541e921dd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.764392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa5a5-55e9-4624-98a4-56db41028aab None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.791358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13490a-50be-4d5b-a9f4-e1541e921dd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:29.792831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13490a-50be-4d5b-a9f4-e1541e921dd2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:29.802813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5d06f-a005-4955-bdb9-a79037e638f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.804415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d06f-a005-4955-bdb9-a79037e638f3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.804910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d06f-a005-4955-bdb9-a79037e638f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.805511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d06f-a005-4955-bdb9-a79037e638f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.806399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5d06f-a005-4955-bdb9-a79037e638f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:29.807008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5d06f-a005-4955-bdb9-a79037e638f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:29.808761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5d06f-a005-4955-bdb9-a79037e638f3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:29.817947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 85/172] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:29.820225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700eeb-695a-4d07-8add-dfa0ba6bd99d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.820834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700eeb-695a-4d07-8add-dfa0ba6bd99d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.821358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700eeb-695a-4d07-8add-dfa0ba6bd99d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.822805 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 86/173] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:29.823702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700eeb-695a-4d07-8add-dfa0ba6bd99d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:29.839144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:29.904139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 88/174] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:29.909041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2697b4-b43c-468a-a690-7f8b7ada30b7 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.909041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2697b4-b43c-468a-a690-7f8b7ada30b7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.909041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2697b4-b43c-468a-a690-7f8b7ada30b7 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.909041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 89/175] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:29.909041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c6c8c-ee72-4ba7-84d0-f2933a43a1c9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:29.909041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c6c8c-ee72-4ba7-84d0-f2933a43a1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:29.909475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c6c8c-ee72-4ba7-84d0-f2933a43a1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:29.942720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-765c6c8c-ee72-4ba7-84d0-f2933a43a1c9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:29.944181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-765c6c8c-ee72-4ba7-84d0-f2933a43a1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:29.957520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700eeb-695a-4d07-8add-dfa0ba6bd99d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:29.963051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:29.998642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:29.999727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.000225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.000783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.001888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=f8f79cdf214b4d86b6f1572f6343002f, user.id=d332b1b14aa24ef8b6ae8d8929faf9ca) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:30.002399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:30.004394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:30.012402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 87/176] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:29 2022] GET /identity/v3/role_assignments?user.id=d332b1b14aa24ef8b6ae8d8929faf9ca&scope.project.id=f8f79cdf214b4d86b6f1572f6343002f&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:30.013613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:30.030793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 90/177] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:30.033513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c6c8c-ee72-4ba7-84d0-f2933a43a1c9 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:30.119983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:30.152176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffa1ea-5092-4544-ab49-5fa8cb58e62f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:30.200084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e678a-81bd-4e95-85ce-24d39108e10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:30.200467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e678a-81bd-4e95-85ce-24d39108e10c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.200747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e678a-81bd-4e95-85ce-24d39108e10c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.201023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e678a-81bd-4e95-85ce-24d39108e10c None admin] PATH_INFO: `/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.203093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e678a-81bd-4e95-85ce-24d39108e10c None admin] RBAC: Authorizing `identity:get_user(user_id=d332b1b14aa24ef8b6ae8d8929faf9ca)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:30.204551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e678a-81bd-4e95-85ce-24d39108e10c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:30.206388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 88/178] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:30 2022] GET /identity/v3/users/d332b1b14aa24ef8b6ae8d8929faf9ca => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:30.213235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412165ee-d509-4033-96ea-427160a56a1c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.213532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412165ee-d509-4033-96ea-427160a56a1c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.213800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412165ee-d509-4033-96ea-427160a56a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.248154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-412165ee-d509-4033-96ea-427160a56a1c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:30.249414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-412165ee-d509-4033-96ea-427160a56a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:30.289704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c6c8c-ee72-4ba7-84d0-f2933a43a1c9 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:30.337563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:30.338235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.338504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.338774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.341691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:30.343196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:30.353399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 91/179] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:30 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:30.354443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:30.379099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:30.384800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 89/180] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:30.385767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412165ee-d509-4033-96ea-427160a56a1c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:30.480874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b1cc-6a63-41bb-b709-fcb0d6cfe8cf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:30.526350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:30.527881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ff24b-1c4f-497a-84ec-ae5e11916d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:30.528520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ff24b-1c4f-497a-84ec-ae5e11916d2a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.528799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ff24b-1c4f-497a-84ec-ae5e11916d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.529068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ff24b-1c4f-497a-84ec-ae5e11916d2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.579459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412165ee-d509-4033-96ea-427160a56a1c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:30.621231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ff24b-1c4f-497a-84ec-ae5e11916d2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:30.622789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ff24b-1c4f-497a-84ec-ae5e11916d2a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:30.656707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:30.671666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:30.671892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.672065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.672335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.673083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=259c064f989f4405a7a84f397fec6223) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:30.673442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:30.674927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:30.685141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 90/181] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:30 2022] GET /identity/v3/role_assignments?user.id=259c064f989f4405a7a84f397fec6223&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:30.687363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:30.767558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 92/182] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:30.769218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20843a8b-405e-4b69-9c27-77a9354e975d None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.769472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20843a8b-405e-4b69-9c27-77a9354e975d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.769779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20843a8b-405e-4b69-9c27-77a9354e975d None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.774597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 93/183] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:30.775760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20843a8b-405e-4b69-9c27-77a9354e975d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:30.816114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d0d69-be46-499a-8fa1-67d21d0672b9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:30.855915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27251ec3-7e39-4695-9dc7-89f9d98b52eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:30.861961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27251ec3-7e39-4695-9dc7-89f9d98b52eb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.861961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27251ec3-7e39-4695-9dc7-89f9d98b52eb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.861961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27251ec3-7e39-4695-9dc7-89f9d98b52eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.861961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:30.870670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27251ec3-7e39-4695-9dc7-89f9d98b52eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-27251ec3-7e39-4695-9dc7-89f9d98b52eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:30.875707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:30.876571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:30.877363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:30.881478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 91/184] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:30.883645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f384d-278e-4d67-bfc1-f526536d2479 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.883926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f384d-278e-4d67-bfc1-f526536d2479 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.884200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f384d-278e-4d67-bfc1-f526536d2479 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.885785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 92/185] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:30.886287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f384d-278e-4d67-bfc1-f526536d2479 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:30.912001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20843a8b-405e-4b69-9c27-77a9354e975d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:30.932292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:30.954731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07069187-5f8f-4db2-8f91-da8efb726448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:30.955373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069187-5f8f-4db2-8f91-da8efb726448 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.955655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069187-5f8f-4db2-8f91-da8efb726448 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.955935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069187-5f8f-4db2-8f91-da8efb726448 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:30.972230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07069187-5f8f-4db2-8f91-da8efb726448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-07069187-5f8f-4db2-8f91-da8efb726448 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:30.978298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:30.979358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:30.980136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:30.982011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 94/186] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:30.983241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe1adf-3c11-46d0-bb75-fbf51dc81cca None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:30.983536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe1adf-3c11-46d0-bb75-fbf51dc81cca None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:30.983812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe1adf-3c11-46d0-bb75-fbf51dc81cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.001628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-fcfe1adf-3c11-46d0-bb75-fbf51dc81cca None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:31.002861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fcfe1adf-3c11-46d0-bb75-fbf51dc81cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:31.004767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f384d-278e-4d67-bfc1-f526536d2479 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.038065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.039139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.039422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.039691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.040808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.042297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.043907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 93/187] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:30 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:31.044662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.071808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 95/188] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:31.073040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2920-8c9c-42fa-8091-2982f22d43db None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.073324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2920-8c9c-42fa-8091-2982f22d43db None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.073596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb2920-8c9c-42fa-8091-2982f22d43db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.091779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-34bb2920-8c9c-42fa-8091-2982f22d43db None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:31.092973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-34bb2920-8c9c-42fa-8091-2982f22d43db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:31.118640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.156867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d76977-1151-410c-8a89-fc7b51960604 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.162211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 96/189] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:31.163668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2920-8c9c-42fa-8091-2982f22d43db None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.199527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.200696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.201395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.201880 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.202943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:31.203395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.204907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.211803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.214636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 94/190] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:31.220156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.291530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb2920-8c9c-42fa-8091-2982f22d43db None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.332954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.333915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.334290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.334675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.336273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:31.336560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.338140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.341929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf631be-67b6-424e-8687-0119547442b7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.351488 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 97/191] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:31.352203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.373053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.381798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.383489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.383777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.384053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.384795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] RBAC: Adding query filter params (role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=d332b1b14aa24ef8b6ae8d8929faf9ca) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:31.385106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.386577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.392488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 95/192] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/role_assignments?user.id=d332b1b14aa24ef8b6ae8d8929faf9ca&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:31.393151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.476835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.481406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cadd4e-1edd-4fe0-b0e1-688b198c0205 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.513310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521985c-3715-49c4-8863-227142bfb518 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.517947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.518599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.518929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.519207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] PATH_INFO: `/v3/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.520398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=cdaf633bc1d0459aaa2b9f2e4ba9edf0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.521942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.529613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 98/193] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/users/cdaf633bc1d0459aaa2b9f2e4ba9edf0 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:31.530465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.555457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.555686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.555850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.556126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.557316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.558827 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.560427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 96/194] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:31.561213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.629228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.654518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be82bb1-d4b5-488c-a7b5-6ab32ca87bb5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.675289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833e4a-9d0e-4d7a-8fd2-c347a1869566 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.688500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.689132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.689411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.689678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] PATH_INFO: `/v3/users/be90f807133e48b0a36f070023218a76` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.703737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=be90f807133e48b0a36f070023218a76)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.705218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.706901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 99/195] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/users/be90f807133e48b0a36f070023218a76 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:31.707664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.714264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.714904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.715186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.715461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.726334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:31.731812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:31.732715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:31.733499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 97/196] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:31.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:31.735363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.809093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4cdd88-d0e9-4467-993b-b249e065a6e6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.833662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3084-93f6-4f7b-99bc-d806076b8b7c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:31.842281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.842891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.843171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.843444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.850348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.867738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:31.868365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:31.868652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:31.868928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:31.869585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.871159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.910279 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:31.911737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:31.935623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:31.992075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 98/197] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:45:31 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:31.993092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.013320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 100/198] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:32.013320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.120442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac132dc-e277-4aab-87d4-5b2a2cd3925e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.128743 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f3a2f-4f97-4791-badb-e14bb4317943 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.157005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.157256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:32.157615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.157928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.158279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.172243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.172857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.173456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.174173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:32.174474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.175851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.185106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 101/199] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:32.185945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.216713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:32.245322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.246848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.313363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51868a-fd8a-404c-afe2-3b879b1f5adf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.359305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.359925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.360207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.360479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.361608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.363106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.364769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 102/200] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:32.365544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.386723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 99/201] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:32.387588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.437338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:32.477590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:32.489192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd754-26c3-4a70-8c3b-b208a9f167c6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.504664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e426ea3-a11d-43a0-8ef7-f3ef06a204e1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.526924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.527572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.527853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.528129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] PATH_INFO: `/v3/users/be90f807133e48b0a36f070023218a76` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.529445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] RBAC: Authorizing `identity:get_user(user_id=be90f807133e48b0a36f070023218a76)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.531032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.532906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 103/202] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/users/be90f807133e48b0a36f070023218a76 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:32.533621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.546710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.547330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.547608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.547981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.548721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] RBAC: Adding query filter params (role.id=8f099e1c603944dfb66a49bef8b0b790, scope.project.id=7a1d4145c5484035a6842313c2648cc1, user.id=cdaf633bc1d0459aaa2b9f2e4ba9edf0) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:32.549031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.550621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.557695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 100/203] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/role_assignments?user.id=cdaf633bc1d0459aaa2b9f2e4ba9edf0&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=8f099e1c603944dfb66a49bef8b0b790 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:32.558534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.651531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518892f-1b4a-4486-811a-35afba52d99a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.678291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:32.684353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f6400-0f37-4f34-b814-d8d821b462d6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.707228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.707894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.708225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.708645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.709991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.711494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.713681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 104/204] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:32.720272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.720429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.720895 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.721177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.721450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.732127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:32.797984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.799578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.927930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102e18e-b922-42a6-a776-66050757fb6d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:32.973835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 101/205] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:32.981483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:32.982475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:32.983100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:32.983709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:32.983989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:32.984737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=be90f807133e48b0a36f070023218a76) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:32.985045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:32.987420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:32.994601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 105/206] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/role_assignments?user.id=be90f807133e48b0a36f070023218a76&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:33.045412 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:33.121829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced16f6-a11d-4213-93c1-640a0a19b0f0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:33.165665 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f83e1-6b7c-46c4-b726-71c5be79f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:33.166753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f83e1-6b7c-46c4-b726-71c5be79f6da None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:33.167257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f83e1-6b7c-46c4-b726-71c5be79f6da None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:33.167726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f83e1-6b7c-46c4-b726-71c5be79f6da None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:33.169236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f83e1-6b7c-46c4-b726-71c5be79f6da None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:33.171010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f83e1-6b7c-46c4-b726-71c5be79f6da None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:33.173241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 102/207] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:32 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:33.180166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:33.249914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:33.296912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70d246-d80a-489a-9633-f00732fd3dad None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:33.331810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b6106-d018-4c92-83be-c745d3be28aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:33.332416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b6106-d018-4c92-83be-c745d3be28aa None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:33.332694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b6106-d018-4c92-83be-c745d3be28aa None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:33.332969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b6106-d018-4c92-83be-c745d3be28aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:33.333860 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b6106-d018-4c92-83be-c745d3be28aa None admin] RBAC: Adding query filter params (role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=be90f807133e48b0a36f070023218a76) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:33.334173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b6106-d018-4c92-83be-c745d3be28aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:33.335904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b6106-d018-4c92-83be-c745d3be28aa None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:33.341838 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 106/208] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:33 2022] GET /identity/v3/role_assignments?user.id=be90f807133e48b0a36f070023218a76&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:34.394922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffe074-0d40-4b74-9830-ac4d7bbd2d22 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:34.395474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffe074-0d40-4b74-9830-ac4d7bbd2d22 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:34.395474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffe074-0d40-4b74-9830-ac4d7bbd2d22 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:34.396905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 103/209] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:34.403863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba1ce8-93c2-4880-9d26-bf02fcff353b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:34.404816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba1ce8-93c2-4880-9d26-bf02fcff353b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:34.405285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba1ce8-93c2-4880-9d26-bf02fcff353b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:34.425578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-68ba1ce8-93c2-4880-9d26-bf02fcff353b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:34.427190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-68ba1ce8-93c2-4880-9d26-bf02fcff353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:34.500659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 107/210] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:34.520499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04932e06-a17c-4c05-8cee-371b090d33bb None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:34.521104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04932e06-a17c-4c05-8cee-371b090d33bb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:34.521576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04932e06-a17c-4c05-8cee-371b090d33bb None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:34.523994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 104/211] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:34.529622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47dfc3-39ca-4704-aec3-7b5f6cfe905d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:34.530195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47dfc3-39ca-4704-aec3-7b5f6cfe905d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:34.530668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47dfc3-39ca-4704-aec3-7b5f6cfe905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:34.550291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-bb47dfc3-39ca-4704-aec3-7b5f6cfe905d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:34.551734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bb47dfc3-39ca-4704-aec3-7b5f6cfe905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:34.597643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:34.621094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 108/212] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:34.777179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04932e06-a17c-4c05-8cee-371b090d33bb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:34.885153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04932e06-a17c-4c05-8cee-371b090d33bb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:34.893128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:34.897363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47dfc3-39ca-4704-aec3-7b5f6cfe905d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:34.922919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:34.923883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:34.924400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:34.924832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:34.935640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:34.941237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:34.942152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:34.942954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:34.946531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 105/213] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:34.951652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:35.011156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47dfc3-39ca-4704-aec3-7b5f6cfe905d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:35.047150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:35.048102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.048570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.049026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.050387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:35.052026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:35.058447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 109/214] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:35.060028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:35.064513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7fc0-f772-499f-9a98-6f5dc3df4361 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:35.103936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:35.104552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.104834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.105106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.105806 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:35.106113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:35.107496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:35.112000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:35.116919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 106/215] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:35.123102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:35.157498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:35.173017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5b237-45a7-4f96-bea6-29ec109370f4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:35.213910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d1ec4-bfb1-4e5f-8b13-74ae16726f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:35.214933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d1ec4-bfb1-4e5f-8b13-74ae16726f47 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.215429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d1ec4-bfb1-4e5f-8b13-74ae16726f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.215950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d1ec4-bfb1-4e5f-8b13-74ae16726f47 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.216927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d1ec4-bfb1-4e5f-8b13-74ae16726f47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:35.218787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d1ec4-bfb1-4e5f-8b13-74ae16726f47 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:35.245082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b01a9-c6ed-46fb-a1d8-19d4db030688 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:35.279558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:35.280191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.280474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.280752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] PATH_INFO: `/v3/users/5ac2eda66ad1493a96e74270559fa160` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.294037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] RBAC: Authorizing `identity:get_user(user_id=5ac2eda66ad1493a96e74270559fa160)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:35.295512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:35.297394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 107/216] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:35 2022] GET /identity/v3/users/5ac2eda66ad1493a96e74270559fa160 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:35.302618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:35.354761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 110/217] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:45:35 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:45:35.356009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e04a14-2471-43ea-bcd4-3fd1b9420daa None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.356291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e04a14-2471-43ea-bcd4-3fd1b9420daa None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.356562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e04a14-2471-43ea-bcd4-3fd1b9420daa None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.357936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 111/218] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:35.363561 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf7630-9655-45c1-b8b8-ad5f3dd4be43 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.363853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf7630-9655-45c1-b8b8-ad5f3dd4be43 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.364118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf7630-9655-45c1-b8b8-ad5f3dd4be43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.385638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0dbf7630-9655-45c1-b8b8-ad5f3dd4be43 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:35.387070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0dbf7630-9655-45c1-b8b8-ad5f3dd4be43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:35.406599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:35.425705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d17786-a9bb-4eea-bdac-e1e8b848bc3e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:35.465969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:35.466597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.466900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.467202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.487923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 112/219] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:35.556583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:35.558009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:35.623470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de93b9-025f-4e0a-aef9-775d021335e2 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.623752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de93b9-025f-4e0a-aef9-775d021335e2 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.624025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de93b9-025f-4e0a-aef9-775d021335e2 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.625412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 113/220] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:35.633220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ef5cc-5b5d-4e88-af99-e2efac9030eb None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.633220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ef5cc-5b5d-4e88-af99-e2efac9030eb None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.633220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ef5cc-5b5d-4e88-af99-e2efac9030eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.657941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:35.659146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-853ef5cc-5b5d-4e88-af99-e2efac9030eb None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:35.660540 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-853ef5cc-5b5d-4e88-af99-e2efac9030eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:35.685733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 108/221] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 12:45:35.691718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:35.704990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:35.758941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 114/222] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:35.794823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ef5cc-5b5d-4e88-af99-e2efac9030eb None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:35.813359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc194957-5e1a-4b2b-89ae-26798478ad51 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:35.861817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:35.862728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:35.863016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:35.863294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:35.864890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:35.867175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:35.869865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 109/223] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:35 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:35.880134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:35.962041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:35.993945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ef5cc-5b5d-4e88-af99-e2efac9030eb None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.047914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27632bde-7452-42f2-a0bf-bf96ba42e95b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.059792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.060960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.061331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.061696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.073291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:36.079018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:36.079865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:36.080657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 115/224] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:36.081299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.093389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.094318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.094775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.095235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.096138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=5ac2eda66ad1493a96e74270559fa160) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:36.096733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:36.098215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:36.106047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 110/225] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:35 2022] GET /identity/v3/role_assignments?user.id=5ac2eda66ad1493a96e74270559fa160&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:36.106758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.115024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:36.268546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5320b-ae88-4f5c-a5ee-8732eae774f0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.299379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca2248-59e4-436b-b034-64cd29f7a8b6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.324280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:36.335612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.336251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.336664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.340141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.346759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.348089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.348574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.349039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.350191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:36.351175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:36.351688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:36.353263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:36.356673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:36.357553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:36.358397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 116/226] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:36.369805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.376890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 111/227] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:36.383113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.520868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36920fd7-c88c-49cb-b25a-f0450108d8a2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.542303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:36.562429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115cff6-5737-4682-9252-23126232d980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.563786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115cff6-5737-4682-9252-23126232d980 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.564073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115cff6-5737-4682-9252-23126232d980 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.564355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115cff6-5737-4682-9252-23126232d980 None admin] PATH_INFO: `/v3/users/be90f807133e48b0a36f070023218a76` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.566153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115cff6-5737-4682-9252-23126232d980 None admin] RBAC: Authorizing `identity:get_user(user_id=be90f807133e48b0a36f070023218a76)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:36.567610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115cff6-5737-4682-9252-23126232d980 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:36.569515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 112/228] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:36 2022] GET /identity/v3/users/be90f807133e48b0a36f070023218a76 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:36.575821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115cff6-5737-4682-9252-23126232d980 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.576266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7dbb-5fa0-4b6d-93e8-6ace2bc42faa None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.655071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.655711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.655993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.656269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.656964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:36.657271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:36.658725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:36.680715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 117/229] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:36.683080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.700088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:36.765315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115cff6-5737-4682-9252-23126232d980 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.811325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.811965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.812239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.812515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.813659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:36.815126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:36.816910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 113/230] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:36 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:36.822840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.879769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019548bf-55ad-43fc-be50-a3623ef49c30 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:36.945530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:36.946198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:36.946472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:36.946740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] PATH_INFO: `/v3/users/be90f807133e48b0a36f070023218a76` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:36.948470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=be90f807133e48b0a36f070023218a76)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:36.949932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:36.956428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 118/231] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:36 2022] GET /identity/v3/users/be90f807133e48b0a36f070023218a76 => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:36.962194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:36.985936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:37.004600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ab04f-da13-4335-9373-72ab1c0f2679 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:37.022135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:37.041813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:37.042552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:37.042840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:37.043119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/be90f807133e48b0a36f070023218a76/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:37.045232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1a6b26bf495458796a0a05b1b3d03ea, user_id=be90f807133e48b0a36f070023218a76, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:37.046791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:37.115381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d377e-1c87-4b5b-8a10-c4e7a3e46bc4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:37.120006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 114/232] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:36 2022] PUT /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/be90f807133e48b0a36f070023218a76/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 12:45:37.162149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c8d70-e4f1-4a47-94bc-c6d5c598f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:37.162728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c8d70-e4f1-4a47-94bc-c6d5c598f756 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:37.163539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c8d70-e4f1-4a47-94bc-c6d5c598f756 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:37.163875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c8d70-e4f1-4a47-94bc-c6d5c598f756 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:37.165069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c8d70-e4f1-4a47-94bc-c6d5c598f756 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:37.166878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c8d70-e4f1-4a47-94bc-c6d5c598f756 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:37.168565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 119/233] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:36 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:37.179166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:37.350252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7fbe6-34d5-4f50-9c4f-cde3084daf45 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:37.392006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc9107-0006-4f0e-8d0e-e6a65bde3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:37.392682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9107-0006-4f0e-8d0e-e6a65bde3477 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:37.392961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9107-0006-4f0e-8d0e-e6a65bde3477 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:37.393235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc9107-0006-4f0e-8d0e-e6a65bde3477 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/be90f807133e48b0a36f070023218a76/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:37.395469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc9107-0006-4f0e-8d0e-e6a65bde3477 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1a6b26bf495458796a0a05b1b3d03ea, user_id=be90f807133e48b0a36f070023218a76, role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:37.396922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc9107-0006-4f0e-8d0e-e6a65bde3477 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:37.495971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 115/234] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:37 2022] PUT /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/be90f807133e48b0a36f070023218a76/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:38.044067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7203e7f-6df1-436b-8e06-284b0bf67252 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:38.044586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7203e7f-6df1-436b-8e06-284b0bf67252 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:38.044586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7203e7f-6df1-436b-8e06-284b0bf67252 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:38.046166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 120/235] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:38.051901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b0d70-d706-4be9-aba0-3b793f49446a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:38.052378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b0d70-d706-4be9-aba0-3b793f49446a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:38.052451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b0d70-d706-4be9-aba0-3b793f49446a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:38.088495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bf0b0d70-d706-4be9-aba0-3b793f49446a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:38.089702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bf0b0d70-d706-4be9-aba0-3b793f49446a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:38.134485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:38.191522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 116/236] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:38.204397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac2381-d612-4db6-8c49-9266d7aa7f3d None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:38.204693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac2381-d612-4db6-8c49-9266d7aa7f3d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:38.204944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac2381-d612-4db6-8c49-9266d7aa7f3d None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:38.206739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 121/237] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:38.223711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dca7f2-37f3-4411-a954-9682ee4bfcca None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:38.224001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dca7f2-37f3-4411-a954-9682ee4bfcca None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:38.224289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dca7f2-37f3-4411-a954-9682ee4bfcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:38.254278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-56dca7f2-37f3-4411-a954-9682ee4bfcca None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:38.255479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-56dca7f2-37f3-4411-a954-9682ee4bfcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:38.406142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 117/238] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:38.563512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac2381-d612-4db6-8c49-9266d7aa7f3d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:38.656988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:38.684557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac2381-d612-4db6-8c49-9266d7aa7f3d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:38.730825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:38.731698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:38.732146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:38.732580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:38.749237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:38.754596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:38.755661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:38.757273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:38.760653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 122/239] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:38.765730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dca7f2-37f3-4411-a954-9682ee4bfcca None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:38.873603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:38.879795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dca7f2-37f3-4411-a954-9682ee4bfcca None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:38.908986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:38.917569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00bbb8-b508-457c-80da-312bae4b7863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:38.918331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00bbb8-b508-457c-80da-312bae4b7863 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:38.918624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00bbb8-b508-457c-80da-312bae4b7863 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:38.918884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00bbb8-b508-457c-80da-312bae4b7863 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:38.919554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00bbb8-b508-457c-80da-312bae4b7863 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:38.919855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00bbb8-b508-457c-80da-312bae4b7863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:38.921235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00bbb8-b508-457c-80da-312bae4b7863 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:38.934245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 118/240] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:38.935457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe06817-66c1-4ccf-9a6b-15664c1a91be None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:38.935742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe06817-66c1-4ccf-9a6b-15664c1a91be None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:38.936014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe06817-66c1-4ccf-9a6b-15664c1a91be None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:38.937286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 119/241] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:38.942217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe06817-66c1-4ccf-9a6b-15664c1a91be None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:39.061703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe06817-66c1-4ccf-9a6b-15664c1a91be None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:39.100382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92858d79-82f5-4864-bf39-a660a6898f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:39.102122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858d79-82f5-4864-bf39-a660a6898f8a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.102868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858d79-82f5-4864-bf39-a660a6898f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.103414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858d79-82f5-4864-bf39-a660a6898f8a None admin] PATH_INFO: `/v3/users/d45ba0fc41ca495e9eecc9976ac9da29` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.112144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09850c93-163e-42b7-af24-9d6fb87aa088 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:39.121253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92858d79-82f5-4864-bf39-a660a6898f8a None admin] RBAC: Authorizing `identity:get_user(user_id=d45ba0fc41ca495e9eecc9976ac9da29)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:39.122939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92858d79-82f5-4864-bf39-a660a6898f8a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:39.124986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 120/242] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:38 2022] GET /identity/v3/users/d45ba0fc41ca495e9eecc9976ac9da29 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:39.126251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb042d6-b616-4c68-be2e-c3ceea7e138a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.129838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb042d6-b616-4c68-be2e-c3ceea7e138a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.129838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb042d6-b616-4c68-be2e-c3ceea7e138a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.151560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3bb042d6-b616-4c68-be2e-c3ceea7e138a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:39.152779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3bb042d6-b616-4c68-be2e-c3ceea7e138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:39.160191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:39.161195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.161701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.162752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.165443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:39.166965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:39.168703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:39.170697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 123/243] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:39.172247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:39.187246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:39.245426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 121/244] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:39.246850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb042d6-b616-4c68-be2e-c3ceea7e138a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:39.299387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0498021-3469-46b5-9a8a-1d0ee648b988 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:39.335787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:39.336394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.336744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.337090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.374959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb042d6-b616-4c68-be2e-c3ceea7e138a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:39.409060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:39.410976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:39.417529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:39.418421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.418881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.419327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.420683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:39.422658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:39.423523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:39.514828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 122/245] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:45:39 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:39.529801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 124/246] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:39.535860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:39.539363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:39.649124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:39.652027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ab79-10ec-4b15-bc96-b12ef494f9c5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:39.735637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:39.736316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.736599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.736883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.738266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:39.739783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:39.744255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 123/247] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:39 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:39.749383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:39.839055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc9c44-eff5-4717-a1e0-2a11b24bad45 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:39.852591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:39.855311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:39.882088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f2166-2ae2-47ad-8d4a-df4e1eb6ef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:39.882950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f2166-2ae2-47ad-8d4a-df4e1eb6ef96 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.883231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f2166-2ae2-47ad-8d4a-df4e1eb6ef96 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.883504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f2166-2ae2-47ad-8d4a-df4e1eb6ef96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.895477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f2166-2ae2-47ad-8d4a-df4e1eb6ef96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:39.895902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f5748-582d-48b8-ad00-ae753ed28013 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c53f2166-2ae2-47ad-8d4a-df4e1eb6ef96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:39.901626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:39.908984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 125/248] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:45:39.909821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a8ce7-1252-4fe4-99ec-ec030dab63a3 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.910549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a8ce7-1252-4fe4-99ec-ec030dab63a3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.910549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a8ce7-1252-4fe4-99ec-ec030dab63a3 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.910549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 126/249] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:39.912290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a8ce7-1252-4fe4-99ec-ec030dab63a3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:39.943180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e9148-92fb-487e-a309-cdeda305b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:39.944938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9148-92fb-487e-a309-cdeda305b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.945815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9148-92fb-487e-a309-cdeda305b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.946119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9148-92fb-487e-a309-cdeda305b1f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:39.946938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e9148-92fb-487e-a309-cdeda305b1f7 None admin] RBAC: Adding query filter params (role.id=8f099e1c603944dfb66a49bef8b0b790, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=d45ba0fc41ca495e9eecc9976ac9da29) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:39.947255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e9148-92fb-487e-a309-cdeda305b1f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:39.948691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e9148-92fb-487e-a309-cdeda305b1f7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:39.955199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 124/250] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:39 2022] GET /identity/v3/role_assignments?user.id=d45ba0fc41ca495e9eecc9976ac9da29&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=8f099e1c603944dfb66a49bef8b0b790 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:39.956400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b119-86c2-4fb3-8ac4-a8a34641920f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:39.956682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b119-86c2-4fb3-8ac4-a8a34641920f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:39.956951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b119-86c2-4fb3-8ac4-a8a34641920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:40.003163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a884b119-86c2-4fb3-8ac4-a8a34641920f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:40.004321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a884b119-86c2-4fb3-8ac4-a8a34641920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:40.042108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a8ce7-1252-4fe4-99ec-ec030dab63a3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:40.087588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:40.088212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:40.088513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:40.088784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:40.089461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:40.089795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:40.091287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:40.103277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 127/251] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:40.108910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:40.126653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 125/252] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:40.150135 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:40.227280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d32012-0856-4672-aec4-90414959fe5a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:40.270359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:40.273763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:40.273763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:40.273763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] PATH_INFO: `/v3/users/5ac2eda66ad1493a96e74270559fa160` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:40.273763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] RBAC: Authorizing `identity:get_user(user_id=5ac2eda66ad1493a96e74270559fa160)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:40.274713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:40.276658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 128/253] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:40 2022] GET /identity/v3/users/5ac2eda66ad1493a96e74270559fa160 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:40.285763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884b119-86c2-4fb3-8ac4-a8a34641920f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:40.353765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:40.393775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884b119-86c2-4fb3-8ac4-a8a34641920f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:40.425717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:40.426387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:40.429758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:40.429758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:40.429758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:40.429758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:40.431252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 126/254] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:40 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:40.437448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:40.551094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:40.555365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbf5f9-892a-4a90-a2d8-540af2862da5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:40.592417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a758ee4-12a3-478c-89ee-e30ffc434364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:40.593079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a758ee4-12a3-478c-89ee-e30ffc434364 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:40.593355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a758ee4-12a3-478c-89ee-e30ffc434364 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:40.593627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a758ee4-12a3-478c-89ee-e30ffc434364 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/5ac2eda66ad1493a96e74270559fa160/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:40.595635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a758ee4-12a3-478c-89ee-e30ffc434364 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1a6b26bf495458796a0a05b1b3d03ea, user_id=5ac2eda66ad1493a96e74270559fa160, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:40.597061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a758ee4-12a3-478c-89ee-e30ffc434364 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:40.677257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 129/255] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:40 2022] PUT /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/5ac2eda66ad1493a96e74270559fa160/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:40.695964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:40.719621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f12032-107d-4626-8c69-0503e9b7f674 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:40.719927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f12032-107d-4626-8c69-0503e9b7f674 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:40.720176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f12032-107d-4626-8c69-0503e9b7f674 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:40.721721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 130/256] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:40.728535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d28989-d221-45f2-af3b-f1dbe84ee4bc None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:40.728827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d28989-d221-45f2-af3b-f1dbe84ee4bc None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:40.729080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d28989-d221-45f2-af3b-f1dbe84ee4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:40.790902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-61d28989-d221-45f2-af3b-f1dbe84ee4bc None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:40.792510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-61d28989-d221-45f2-af3b-f1dbe84ee4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:40.939355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec9e34-995d-4400-b745-0be3c9b6e2f2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:40.984728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:40.991993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:40.992749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:40.993162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:40.993398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:41.009389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:41.015451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:41.016450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:41.017303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 127/257] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:41.021727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:41.040472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 131/258] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:41.137474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422ae02-5923-4f51-87d3-635c42972de4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:41.173431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:41.174075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:41.174356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:41.174623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:41.175293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:41.175591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:41.176954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:41.186926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 128/259] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:41.195242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d28989-d221-45f2-af3b-f1dbe84ee4bc None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:41.306133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:41.318416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d28989-d221-45f2-af3b-f1dbe84ee4bc None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:41.354465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:41.355344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:41.355764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:41.356203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] PATH_INFO: `/v3/users/be90f807133e48b0a36f070023218a76` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:41.357481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] RBAC: Authorizing `identity:get_user(user_id=be90f807133e48b0a36f070023218a76)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:41.359168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:41.361461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 132/260] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:41 2022] GET /identity/v3/users/be90f807133e48b0a36f070023218a76 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:41.370328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:41.428206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:41.476552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646161fe-cab4-4f1f-b436-7a533aa3eb17 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:41.511476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:41.512338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:41.512758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:41.513167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:41.588401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:41.590125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:41.654127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:41.695068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 129/261] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:41.698894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:41.716584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:41.848382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525263c-ee61-4170-bd32-7699f830ce18 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:41.879736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00193d1-6a96-465c-bbce-8148b8b85170 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:41.886918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:41.887872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:41.888289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:41.888701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:41.889999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:41.891563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:41.893339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 130/262] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:41 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:41.899352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:41.925598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:41.945801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:41.946448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:41.947040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:41.947040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:41.969218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:41.974371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:41.975526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:41.976675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 133/263] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:41.981832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:42.017197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ecd03-3e42-4af6-8cf1-bbb8f1226ab8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:42.021186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:42.073999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:42.074655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:42.089557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:42.089557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:42.089557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=be90f807133e48b0a36f070023218a76) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:42.089557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:42.089557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:42.089946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 131/264] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:41 2022] GET /identity/v3/role_assignments?user.id=be90f807133e48b0a36f070023218a76&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:42.181224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383aee8d-1bf9-4c2e-b6d2-82b032145f27 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:42.230676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:42.231283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:42.231544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:42.231816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:42.232487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:42.232783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:42.234288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:42.243576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 134/265] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:42.249413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:42.381215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:42.401434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7033e1f-755a-4859-a787-d438363032b0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:42.442427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:42.442982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:42.443309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:42.443585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] PATH_INFO: `/v3/users/be90f807133e48b0a36f070023218a76` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:42.444774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] RBAC: Authorizing `identity:get_user(user_id=be90f807133e48b0a36f070023218a76)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:42.446240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:42.448124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 132/266] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:42 2022] GET /identity/v3/users/be90f807133e48b0a36f070023218a76 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:42.453805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:42.547270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:42.569662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf17b9f-2aca-4abd-9971-5b7621b66d8c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:42.610875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:42.611835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:42.612342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:42.612824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:42.691302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:42.692834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:42.774976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:42.798796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 135/267] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:42.804810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:42.969471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc7f7b-4385-4984-81f6-c131ba3ee8b2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:43.017128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650e6a2-df56-4400-90a9-a96ff71828d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:43.017784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650e6a2-df56-4400-90a9-a96ff71828d6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.018067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650e6a2-df56-4400-90a9-a96ff71828d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.018336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650e6a2-df56-4400-90a9-a96ff71828d6 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.019468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650e6a2-df56-4400-90a9-a96ff71828d6 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:43.020887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650e6a2-df56-4400-90a9-a96ff71828d6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:43.022846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 133/268] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:42 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:43.028955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:43.172086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2f328-1b48-4793-b175-a99d4e2a4d32 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:43.213863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac030db-426d-4836-b653-6bebfa8685a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:43.214481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac030db-426d-4836-b653-6bebfa8685a8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.214761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac030db-426d-4836-b653-6bebfa8685a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.221289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac030db-426d-4836-b653-6bebfa8685a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.221289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac030db-426d-4836-b653-6bebfa8685a8 None admin] RBAC: Adding query filter params (role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=be90f807133e48b0a36f070023218a76) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:43.221289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac030db-426d-4836-b653-6bebfa8685a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:43.221289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac030db-426d-4836-b653-6bebfa8685a8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:43.229832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 136/269] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:43 2022] GET /identity/v3/role_assignments?user.id=be90f807133e48b0a36f070023218a76&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:43.343877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335af955-c832-46a1-9f2e-53f96f340d06 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.344522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335af955-c832-46a1-9f2e-53f96f340d06 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.344522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335af955-c832-46a1-9f2e-53f96f340d06 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.346095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 134/270] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:43.351467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb848-df8b-4681-85d3-63d44e3ceb65 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.352070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb848-df8b-4681-85d3-63d44e3ceb65 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.352070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb848-df8b-4681-85d3-63d44e3ceb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.376073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-98cbb848-df8b-4681-85d3-63d44e3ceb65 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:43.377354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-98cbb848-df8b-4681-85d3-63d44e3ceb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:43.429778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93e714-d086-457d-80de-5571a49b0735 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.429778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93e714-d086-457d-80de-5571a49b0735 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.429778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93e714-d086-457d-80de-5571a49b0735 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.430396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:43.431335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 135/271] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:43.439484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0efa9-3c7c-491f-90ba-c2082b85cee8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.439765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0efa9-3c7c-491f-90ba-c2082b85cee8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.447747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0efa9-3c7c-491f-90ba-c2082b85cee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.468750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-17d0efa9-3c7c-491f-90ba-c2082b85cee8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:43.469046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 137/272] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:43.470293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-17d0efa9-3c7c-491f-90ba-c2082b85cee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:43.489567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:43.502440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92340b-08b6-4fd6-bc7e-c5d8a82d8327 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.503002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92340b-08b6-4fd6-bc7e-c5d8a82d8327 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.503444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92340b-08b6-4fd6-bc7e-c5d8a82d8327 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.505514 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 138/273] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:43.510943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa1d7a-3c6c-4c73-8ecd-070e26dd4918 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.511221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa1d7a-3c6c-4c73-8ecd-070e26dd4918 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.511490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa1d7a-3c6c-4c73-8ecd-070e26dd4918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.530113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-84aa1d7a-3c6c-4c73-8ecd-070e26dd4918 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:43.531348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-84aa1d7a-3c6c-4c73-8ecd-070e26dd4918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:43.549373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 136/274] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:43.612449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 139/275] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:43.770235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0efa9-3c7c-491f-90ba-c2082b85cee8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:43.851409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88a49-ffc8-4b2f-a6fa-608cbfadefc4 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.852034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88a49-ffc8-4b2f-a6fa-608cbfadefc4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.852508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d88a49-ffc8-4b2f-a6fa-608cbfadefc4 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.854400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 140/276] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:45:43.856043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88a49-ffc8-4b2f-a6fa-608cbfadefc4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:43.878269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0efa9-3c7c-491f-90ba-c2082b85cee8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:43.914272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54433-0b3f-4b78-90c3-67c306eb46e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:43.914897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54433-0b3f-4b78-90c3-67c306eb46e6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.915174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54433-0b3f-4b78-90c3-67c306eb46e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.915442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54433-0b3f-4b78-90c3-67c306eb46e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.926151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54433-0b3f-4b78-90c3-67c306eb46e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-22d54433-0b3f-4b78-90c3-67c306eb46e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:43.931406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:43.932357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:43.933169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 137/277] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:43.934613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df6612-7cd0-41af-ae5d-0348113269fe None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:43.934613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df6612-7cd0-41af-ae5d-0348113269fe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:43.934886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df6612-7cd0-41af-ae5d-0348113269fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:43.954750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-94df6612-7cd0-41af-ae5d-0348113269fe None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:43.955941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-94df6612-7cd0-41af-ae5d-0348113269fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:43.964035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:43.972337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d88a49-ffc8-4b2f-a6fa-608cbfadefc4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:43.973995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.005606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.006316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.007215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.007490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.008584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.010071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.011616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 141/278] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:44.013930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.023631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 138/279] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:44.024420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df6612-7cd0-41af-ae5d-0348113269fe None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.119427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04c090-ecd9-4696-9c01-f3c6c83cd91f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.135070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df6612-7cd0-41af-ae5d-0348113269fe None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.155057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.155681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.155958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.156231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.156891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:44.157195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.158600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.167209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 142/280] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:44.172510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.172645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.173146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.173429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.173701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.185166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:44.190441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:44.191289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:44.192083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 139/281] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:44.192712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.196580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.211305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.282402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775e0e7-d38f-4c71-9bac-35accc09b4a6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.301522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbaf815-3e6e-4525-9849-53b08f43288f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.315741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.316340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.316616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.316882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.317517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.319646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.336605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.337219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.337502 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.342266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] PATH_INFO: `/v3/users/d45ba0fc41ca495e9eecc9976ac9da29` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.342266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] RBAC: Authorizing `identity:get_user(user_id=d45ba0fc41ca495e9eecc9976ac9da29)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.342266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.342266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 140/282] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/users/d45ba0fc41ca495e9eecc9976ac9da29 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:44.342718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.406278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 143/283] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:45:44 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:44.407682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.420767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.454865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3c045-fb29-4e10-9ef0-c6c739d0e117 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.493194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.494153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.494578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.494995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.495823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:44.496136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.497533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.511259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 141/284] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:44.511259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.511550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.524776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489969b-e1a7-4bc8-b149-3b341a008ce6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.562839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.563426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.563672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.563946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.576163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:44.582802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:44.583684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 144/285] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:44.584481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.636357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8142916-9c47-413e-a0dc-2e57ae3a75a1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.675790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.682168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.682811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.683096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.683369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.687040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.687040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.693481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 142/286] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:44.693481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.708731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9762-9d3b-47d6-b1ec-f78550b3b6f1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.750357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.750998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.751280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.751555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] PATH_INFO: `/v3/users/a130725bc6f44b2899504bf6e89cf436` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.768780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] RBAC: Authorizing `identity:get_user(user_id=a130725bc6f44b2899504bf6e89cf436)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.770937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.773796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 145/287] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/users/a130725bc6f44b2899504bf6e89cf436 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:44.774521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.785864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.834222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18315f-65a9-49af-8bec-88b659dcd2b9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.870854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.871482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.872083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.872398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.873114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:44.873461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.875330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.886089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 143/288] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:44.886961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:44.898551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d88f0-61a0-42de-a945-c74464a11e9e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:44.939434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:44.940201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:44.940492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:44.940764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/d45ba0fc41ca495e9eecc9976ac9da29/roles/8f099e1c603944dfb66a49bef8b0b790` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:44.942859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] RBAC: Authorizing `identity:create_grant(project_id=e1a6b26bf495458796a0a05b1b3d03ea, user_id=d45ba0fc41ca495e9eecc9976ac9da29, role_id=8f099e1c603944dfb66a49bef8b0b790)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:44.945235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:44.956040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:44.994642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf35a2f-8f30-423c-8522-f453f848b116 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:45.028559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 146/289] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:45:44 2022] PUT /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/users/d45ba0fc41ca495e9eecc9976ac9da29/roles/8f099e1c603944dfb66a49bef8b0b790 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:45.028559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:45.031449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:45.033501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:45.033501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:45.033501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:45.033501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, user.id=a130725bc6f44b2899504bf6e89cf436, scope.system=all) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:45.033954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:45.035372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:45.041965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 144/290] 10.222.0.72 () {56 vars in 1308 bytes} [Tue Sep 13 12:45:44 2022] GET /identity/v3/role_assignments?user.id=a130725bc6f44b2899504bf6e89cf436&scope.system=all&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:45.153989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:45.202085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73ef7-f248-43d8-8167-f59471ab9b5e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:45.247263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:45.247502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:45.247726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:45.247879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] PATH_INFO: `/v3/users/5ac2eda66ad1493a96e74270559fa160` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:45.254657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] RBAC: Authorizing `identity:get_user(user_id=5ac2eda66ad1493a96e74270559fa160)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:45.256169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:45.259063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 147/291] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:45 2022] GET /identity/v3/users/5ac2eda66ad1493a96e74270559fa160 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:45.264277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:45.399890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bb1d7-8530-48ad-81a5-78a7e5a6592b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:45.427521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:45.437762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:45.442163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:45.442163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:45.442163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:45.512950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:45.514480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:45.616385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 145/292] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:45.621824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:45.794775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a18a4-1b36-4000-b110-e7192b3e62f6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:45.839998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd55a2-2d65-4af1-9b5b-f0d2ad19c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:45.840629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd55a2-2d65-4af1-9b5b-f0d2ad19c675 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:45.840911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd55a2-2d65-4af1-9b5b-f0d2ad19c675 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:45.841188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd55a2-2d65-4af1-9b5b-f0d2ad19c675 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:45.842569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd55a2-2d65-4af1-9b5b-f0d2ad19c675 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:45.844022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd55a2-2d65-4af1-9b5b-f0d2ad19c675 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:45.846024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 148/293] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:45 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:45.852035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:45.898212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:45.995555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decb004-e32e-499c-9f78-fa53001caa25 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:46.031030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e4664-5076-421d-a397-0e4d1fe0b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:46.031640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e4664-5076-421d-a397-0e4d1fe0b847 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:46.031924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e4664-5076-421d-a397-0e4d1fe0b847 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:46.032201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e4664-5076-421d-a397-0e4d1fe0b847 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:46.032928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e4664-5076-421d-a397-0e4d1fe0b847 None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=5ac2eda66ad1493a96e74270559fa160) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:46.033239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e4664-5076-421d-a397-0e4d1fe0b847 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:46.034732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e4664-5076-421d-a397-0e4d1fe0b847 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:46.041102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 146/294] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:45 2022] GET /identity/v3/role_assignments?user.id=5ac2eda66ad1493a96e74270559fa160&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:47.033843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846e92f-723a-400b-896c-f98e2e177828 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.034358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846e92f-723a-400b-896c-f98e2e177828 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.034462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846e92f-723a-400b-896c-f98e2e177828 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.036008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 149/295] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:47.042563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e20d-de6f-4832-8782-462c756f1f24 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.043076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e20d-de6f-4832-8782-462c756f1f24 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.043149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e20d-de6f-4832-8782-462c756f1f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.062889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cd86e20d-de6f-4832-8782-462c756f1f24 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:47.064122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cd86e20d-de6f-4832-8782-462c756f1f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:47.138439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 147/296] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:47.176623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bc576-cc2e-40c7-9edf-be4bb8d56fde None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.176924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bc576-cc2e-40c7-9edf-be4bb8d56fde None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.177177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bc576-cc2e-40c7-9edf-be4bb8d56fde None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.179484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 150/297] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:47.184904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95657a-ca8e-488b-bf08-1d791184f3e7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.185362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95657a-ca8e-488b-bf08-1d791184f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.185900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95657a-ca8e-488b-bf08-1d791184f3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.208558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6e95657a-ca8e-488b-bf08-1d791184f3e7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:47.210790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6e95657a-ca8e-488b-bf08-1d791184f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:47.224970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:47.283663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 148/298] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:47.422010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bc576-cc2e-40c7-9edf-be4bb8d56fde None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:47.425005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:47.539043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bc576-cc2e-40c7-9edf-be4bb8d56fde None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:47.541933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30d0be-2998-47e0-b09e-27bbcbc53e7c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.542228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30d0be-2998-47e0-b09e-27bbcbc53e7c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.542511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30d0be-2998-47e0-b09e-27bbcbc53e7c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.545279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 149/299] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:47.549908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868d72-5d59-4d75-a69e-222c143e83ce None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.550185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868d72-5d59-4d75-a69e-222c143e83ce None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.550458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868d72-5d59-4d75-a69e-222c143e83ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.571153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-02868d72-5d59-4d75-a69e-222c143e83ce None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:47.572537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-02868d72-5d59-4d75-a69e-222c143e83ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:47.582994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:47.584002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.584284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.585115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.595938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:47.601251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:47.602200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:47.603026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 151/300] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:47.603694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:47.658857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 150/301] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:47.659711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02868d72-5d59-4d75-a69e-222c143e83ce None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:47.673993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:47.713933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56474fd-7c9f-4b65-922f-f5eff79f4f02 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:47.747321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:47.748954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff51675-a6c4-4a1b-a8d9-468494bb176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:47.749595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff51675-a6c4-4a1b-a8d9-468494bb176d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.749901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff51675-a6c4-4a1b-a8d9-468494bb176d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.750174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff51675-a6c4-4a1b-a8d9-468494bb176d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.751264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff51675-a6c4-4a1b-a8d9-468494bb176d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:47.752696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff51675-a6c4-4a1b-a8d9-468494bb176d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:47.754341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 152/302] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:47.755438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f32168-eef9-4a18-9e56-d1d9427fd9e3 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.755825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f32168-eef9-4a18-9e56-d1d9427fd9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.756098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f32168-eef9-4a18-9e56-d1d9427fd9e3 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.758402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 153/303] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:47.762792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f32168-eef9-4a18-9e56-d1d9427fd9e3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:47.772960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02868d72-5d59-4d75-a69e-222c143e83ce None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:47.815453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe921a-be8e-4533-bb0c-b3d781e17861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:47.816323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe921a-be8e-4533-bb0c-b3d781e17861 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.816777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe921a-be8e-4533-bb0c-b3d781e17861 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.817221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe921a-be8e-4533-bb0c-b3d781e17861 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.818608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe921a-be8e-4533-bb0c-b3d781e17861 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:47.819843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe921a-be8e-4533-bb0c-b3d781e17861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:47.821587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe921a-be8e-4533-bb0c-b3d781e17861 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:47.832400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 151/304] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:47.834076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3ab5e-36c6-4265-84d2-12f9aeffef90 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.835135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3ab5e-36c6-4265-84d2-12f9aeffef90 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.835411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3ab5e-36c6-4265-84d2-12f9aeffef90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.854383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f3f3ab5e-36c6-4265-84d2-12f9aeffef90 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:47.855662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f3f3ab5e-36c6-4265-84d2-12f9aeffef90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:47.870894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f32168-eef9-4a18-9e56-d1d9427fd9e3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:47.901808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:47.908159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:47.908762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:47.909033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:47.909302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] PATH_INFO: `/v3/groups` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:47.910005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:47.911450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:47.929672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 152/305] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:47.930484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3ab5e-36c6-4265-84d2-12f9aeffef90 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:47.980626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 154/306] 10.222.0.72 () {60 vars in 1134 bytes} [Tue Sep 13 12:45:47 2022] POST /identity/v3/groups => generated 229 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:45:47.981670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.012110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:48.047558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3ab5e-36c6-4265-84d2-12f9aeffef90 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.091197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.092083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.092574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.093077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] PATH_INFO: `/v3/users/eef469e465ab4783b35e1be0d9dd8238` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.108629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] RBAC: Authorizing `identity:get_user(user_id=eef469e465ab4783b35e1be0d9dd8238)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:48.110465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:48.112471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 153/307] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:47 2022] GET /identity/v3/users/eef469e465ab4783b35e1be0d9dd8238 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:48.121846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e655c92-4187-480d-839d-5023872eee1b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.123612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.163320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.164064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.164351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.164627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.185112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:48.190547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:48.191397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:48.192170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 155/308] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:48.197637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.255153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:48.262932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8a686-f4ee-41a6-af72-fa659e222f6a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.307169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:48.314017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.314942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.315396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.315851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.316691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] RBAC: Adding query filter params (role.id=8f099e1c603944dfb66a49bef8b0b790, user.id=eef469e465ab4783b35e1be0d9dd8238, scope.system=all) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:48.317188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:48.319887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:48.321833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af4ea-a3c5-4960-baa3-1c9e33d4f4cb None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.325027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 154/309] 10.222.0.72 () {56 vars in 1308 bytes} [Tue Sep 13 12:45:48 2022] GET /identity/v3/role_assignments?user.id=eef469e465ab4783b35e1be0d9dd8238&scope.system=all&role.id=8f099e1c603944dfb66a49bef8b0b790 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:48.325840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.367239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.367939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.368140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.368399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.369077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:48.369377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:48.370893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:48.380132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 156/310] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:48.386622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.481506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762b86f-d7e6-42dd-86cb-56282b4b4a73 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.503625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad8a5-3551-429d-bde9-8b4df427c8ab None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.520188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:48.532315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.533531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.534058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.534406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.546027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.546814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.547219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.547583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] PATH_INFO: `/v3/users/a130725bc6f44b2899504bf6e89cf436` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.548899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] RBAC: Authorizing `identity:get_user(user_id=a130725bc6f44b2899504bf6e89cf436)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:48.549764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:48.550729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:48.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:48.557808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:48.558599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:48.559176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 157/311] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:48 2022] GET /identity/v3/users/a130725bc6f44b2899504bf6e89cf436 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:48.559176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.559294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 155/312] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:48.565411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.628421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:48.682369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6646a-91c5-4053-b1a6-fd6e7c71dbd8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.702247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34485c1d-8d11-4499-8cdc-1c0d6bfbf1e5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.719212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.720144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.720637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.721096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.721994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:48.722485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:48.724077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:48.737975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 156/313] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:48.739814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.742178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.742844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.743122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.743398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] PATH_INFO: `/v3/system/users/a130725bc6f44b2899504bf6e89cf436/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.745016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a130725bc6f44b2899504bf6e89cf436, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:48.747303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:48.845776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 158/314] 10.222.0.72 () {58 vars in 1334 bytes} [Tue Sep 13 12:45:48 2022] PUT /identity/v3/system/users/a130725bc6f44b2899504bf6e89cf436/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 12:45:48.855248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f997ce-2984-4223-aac9-74171dae954c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:48.877387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:48.898641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:48.899307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:48.899591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:48.899871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] PATH_INFO: `/v3/users/d45ba0fc41ca495e9eecc9976ac9da29` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:48.901233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=d45ba0fc41ca495e9eecc9976ac9da29)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:48.902899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:48.904653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 157/315] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:48 2022] GET /identity/v3/users/d45ba0fc41ca495e9eecc9976ac9da29 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:48.910755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:48.968310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:49.035925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88148c-0791-4a20-b345-ab811ba8ae80 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:49.073037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:49.073648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:49.073980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:49.074312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:49.147894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:49.149364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:49.196264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:49.262176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 159/316] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 12:45:49.267800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:49.371534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0b975-34e2-463c-b6ce-644bc378e0e2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:49.407174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9cb1e-d99e-4e0c-9da4-df8a999830a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:49.407812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9cb1e-d99e-4e0c-9da4-df8a999830a6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:49.408091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9cb1e-d99e-4e0c-9da4-df8a999830a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:49.408365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9cb1e-d99e-4e0c-9da4-df8a999830a6 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:49.409499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9cb1e-d99e-4e0c-9da4-df8a999830a6 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:49.410968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9cb1e-d99e-4e0c-9da4-df8a999830a6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:49.414323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 158/317] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:49 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:49.419830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:49.528070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ef479-f6d2-4866-84b7-6019a45937bc None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:49.566543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12630b6e-1a25-402d-98b5-05aa6f86f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:49.567215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630b6e-1a25-402d-98b5-05aa6f86f591 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:49.567504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630b6e-1a25-402d-98b5-05aa6f86f591 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:49.567773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630b6e-1a25-402d-98b5-05aa6f86f591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:49.569668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12630b6e-1a25-402d-98b5-05aa6f86f591 None admin] RBAC: Adding query filter params (role.id=8f099e1c603944dfb66a49bef8b0b790, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea, user.id=d45ba0fc41ca495e9eecc9976ac9da29) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:49.570270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12630b6e-1a25-402d-98b5-05aa6f86f591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:49.572536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12630b6e-1a25-402d-98b5-05aa6f86f591 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:49.580939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 160/318] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:45:49 2022] GET /identity/v3/role_assignments?user.id=d45ba0fc41ca495e9eecc9976ac9da29&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=8f099e1c603944dfb66a49bef8b0b790 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:50.382288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd3e0-ce9d-4738-acb5-ab396daf0902 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:50.382952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd3e0-ce9d-4738-acb5-ab396daf0902 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:50.382952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd3e0-ce9d-4738-acb5-ab396daf0902 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:50.384215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 159/319] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:50.390300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77536-230e-43d0-a207-a11d5b84c2e5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:50.390797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77536-230e-43d0-a207-a11d5b84c2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:50.390867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77536-230e-43d0-a207-a11d5b84c2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:50.410367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-14a77536-230e-43d0-a207-a11d5b84c2e5 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:50.411607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-14a77536-230e-43d0-a207-a11d5b84c2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:50.413997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:50.481680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 161/320] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:50.512080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a856a5-b6f6-413d-8d42-769af714df96 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:50.512390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a856a5-b6f6-413d-8d42-769af714df96 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:50.512639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a856a5-b6f6-413d-8d42-769af714df96 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:50.514132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 160/321] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:50.519079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7a53-be11-49ee-b7ed-04ec85877822 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:50.519368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7a53-be11-49ee-b7ed-04ec85877822 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:50.519868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7a53-be11-49ee-b7ed-04ec85877822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:50.538957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-732a7a53-be11-49ee-b7ed-04ec85877822 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:50.540768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-732a7a53-be11-49ee-b7ed-04ec85877822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:50.610403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 162/322] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:50.751332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a856a5-b6f6-413d-8d42-769af714df96 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:50.804886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:50.853555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a856a5-b6f6-413d-8d42-769af714df96 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:50.880949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a7a53-be11-49ee-b7ed-04ec85877822 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:50.896687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:50.896687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:50.896687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:50.896687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:50.897137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:50.897137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:50.897137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 161/323] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:45:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:50.901901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:50.965431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:51.005704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a7a53-be11-49ee-b7ed-04ec85877822 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:51.009165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b017f-6658-48ab-b8d8-ab4841776845 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:51.032398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:51.046638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:51.048386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e55fe-74cc-42a8-b62f-8a55d7149657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:51.048493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.048581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e55fe-74cc-42a8-b62f-8a55d7149657 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.048673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.048742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e55fe-74cc-42a8-b62f-8a55d7149657 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.048806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.048869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e55fe-74cc-42a8-b62f-8a55d7149657 None admin] PATH_INFO: `/v3/groups` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.048869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e55fe-74cc-42a8-b62f-8a55d7149657 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:51.050104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e55fe-74cc-42a8-b62f-8a55d7149657 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:51.060532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:51.065601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:51.066703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:51.067503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 163/324] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:45:51.073147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:51.171178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e08d55-0678-4870-8cdd-e2ae34a7f49e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:51.180034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 162/325] 10.222.0.72 () {60 vars in 1134 bytes} [Tue Sep 13 12:45:50 2022] POST /identity/v3/groups => generated 226 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:45:51.181544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b9ee9-4fdf-477b-937e-10c22a37a46e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.181851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b9ee9-4fdf-477b-937e-10c22a37a46e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.188888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b9ee9-4fdf-477b-937e-10c22a37a46e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.188888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 163/326] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:51.189302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f840e1-b09a-449f-b076-aac8699db788 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.189589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f840e1-b09a-449f-b076-aac8699db788 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.189900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f840e1-b09a-449f-b076-aac8699db788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.209336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-29f840e1-b09a-449f-b076-aac8699db788 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:51.210624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-29f840e1-b09a-449f-b076-aac8699db788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:51.212908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:51.222424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:51.223159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.226115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.226115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.226115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:51.226115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:51.226705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:51.236167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 164/327] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:51.242646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:51.285677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 164/328] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:51.355751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0eade-1c23-4e40-97bf-72f6451cd5af None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:51.391710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:51.392349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.392604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.392870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] PATH_INFO: `/v3/users/eef469e465ab4783b35e1be0d9dd8238` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.394016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=eef469e465ab4783b35e1be0d9dd8238)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:51.395446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:51.397950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 165/329] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:51 2022] GET /identity/v3/users/eef469e465ab4783b35e1be0d9dd8238 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:51.403396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f840e1-b09a-449f-b076-aac8699db788 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:51.515945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f840e1-b09a-449f-b076-aac8699db788 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:51.528510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:51.553007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:51.553988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.554247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.554528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] PATH_INFO: `/v3/system/users/eef469e465ab4783b35e1be0d9dd8238/roles/8f099e1c603944dfb66a49bef8b0b790` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.555967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=eef469e465ab4783b35e1be0d9dd8238, role_id=8f099e1c603944dfb66a49bef8b0b790)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:51.557357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:51.560635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:51.625676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:51.635258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 165/330] 10.222.0.72 () {58 vars in 1334 bytes} [Tue Sep 13 12:45:51 2022] PUT /identity/v3/system/users/eef469e465ab4783b35e1be0d9dd8238/roles/8f099e1c603944dfb66a49bef8b0b790 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:45:51.686421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3dbe0a-5d80-43d7-a0c7-c4c06eaea3b6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:51.733464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:51.734174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.734458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.734737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.751284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:51.757343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:51.758334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:51.759126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:51.759738 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 166/331] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:45:51.767580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:51.897347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c4867-5342-4d9a-94b9-6e510497adc0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:51.937076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:51.937697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:51.938006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:51.938278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:51.938951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:51.939248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:51.940615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:51.951809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 166/332] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:51.957341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:52.092383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792c95-14c1-4cb1-97d0-455e97e75840 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:52.112541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:52.131788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2f4f9-e7b9-4627-adf8-290c23a64d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:52.132535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f4f9-e7b9-4627-adf8-290c23a64d8c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:52.133055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f4f9-e7b9-4627-adf8-290c23a64d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:52.133406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f4f9-e7b9-4627-adf8-290c23a64d8c None admin] PATH_INFO: `/v3/users/a130725bc6f44b2899504bf6e89cf436` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:52.134740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2f4f9-e7b9-4627-adf8-290c23a64d8c None admin] RBAC: Authorizing `identity:get_user(user_id=a130725bc6f44b2899504bf6e89cf436)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:52.136281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2f4f9-e7b9-4627-adf8-290c23a64d8c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:52.139357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 167/333] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:51 2022] GET /identity/v3/users/a130725bc6f44b2899504bf6e89cf436 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:52.144406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:52.189361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:52.263356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620f0ee-f339-4f17-b488-1692fc276b33 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:52.300504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03b869-48a8-499b-816d-be84595a7652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:52.301199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b869-48a8-499b-816d-be84595a7652 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:52.301549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b869-48a8-499b-816d-be84595a7652 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:52.301927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b869-48a8-499b-816d-be84595a7652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:52.302727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03b869-48a8-499b-816d-be84595a7652 None admin] RBAC: Adding query filter params (role.id=4279c61c3b4245e39915c7d4d0293dfe, user.id=a130725bc6f44b2899504bf6e89cf436, scope.system=all) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:52.303191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03b869-48a8-499b-816d-be84595a7652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:52.305327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03b869-48a8-499b-816d-be84595a7652 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:52.320577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 167/334] 10.222.0.72 () {56 vars in 1308 bytes} [Tue Sep 13 12:45:52 2022] GET /identity/v3/role_assignments?user.id=a130725bc6f44b2899504bf6e89cf436&scope.system=all&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:53.767000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f917a5-3afb-4302-b94b-fc31eb0d1e69 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:53.767525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f917a5-3afb-4302-b94b-fc31eb0d1e69 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:53.767525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f917a5-3afb-4302-b94b-fc31eb0d1e69 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:53.769164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 168/335] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:53.775735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafd911-8335-457f-b422-0c2dba1f9c5b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:53.776185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafd911-8335-457f-b422-0c2dba1f9c5b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:53.776257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafd911-8335-457f-b422-0c2dba1f9c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:53.798750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-baafd911-8335-457f-b422-0c2dba1f9c5b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:53.799963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-baafd911-8335-457f-b422-0c2dba1f9c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:53.887218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 168/336] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:54.439982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50adeb8-6caf-4b4a-9337-ec8c0c02cf6e None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:54.440343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50adeb8-6caf-4b4a-9337-ec8c0c02cf6e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:54.440603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50adeb8-6caf-4b4a-9337-ec8c0c02cf6e None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:54.442282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 169/337] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:54.447427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff6fb-9f12-433a-bfeb-0e4a24aeebea None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:54.447727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff6fb-9f12-433a-bfeb-0e4a24aeebea None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:54.448001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff6fb-9f12-433a-bfeb-0e4a24aeebea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:54.462707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50adeb8-6caf-4b4a-9337-ec8c0c02cf6e None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:54.478802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b76ff6fb-9f12-433a-bfeb-0e4a24aeebea None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:54.480022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b76ff6fb-9f12-433a-bfeb-0e4a24aeebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:54.518130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:54.597968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 169/338] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:54.599514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c0164-c172-4604-9605-d0fc47d1cee4 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:54.599832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c0164-c172-4604-9605-d0fc47d1cee4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:54.601103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c0164-c172-4604-9605-d0fc47d1cee4 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:54.602799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 170/339] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:54.610487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080b76a-9750-49f5-8d92-d78e70923c7e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:54.610768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080b76a-9750-49f5-8d92-d78e70923c7e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:54.611592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080b76a-9750-49f5-8d92-d78e70923c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:54.647723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b080b76a-9750-49f5-8d92-d78e70923c7e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:54.649083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b080b76a-9750-49f5-8d92-d78e70923c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:54.670417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50adeb8-6caf-4b4a-9337-ec8c0c02cf6e None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:54.737057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:54.737700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:54.738011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:54.738280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:54.759359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:54.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:54.765143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:54.765960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:54.766552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 170/340] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:54.774317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:54.781332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 171/341] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:45:54.841775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:54.904168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64121611-4461-48cd-a63f-f15d0d80a8e6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:54.950060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:54.950673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:54.950947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:54.951219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:54.952143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:54.952438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:54.953885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:54.964700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 171/342] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:54.970444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080b76a-9750-49f5-8d92-d78e70923c7e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:55.064991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:55.087604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080b76a-9750-49f5-8d92-d78e70923c7e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:55.130457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ab90d-703d-47ee-bc05-dda2bc242a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:55.131722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ab90d-703d-47ee-bc05-dda2bc242a67 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.132001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ab90d-703d-47ee-bc05-dda2bc242a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.132276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ab90d-703d-47ee-bc05-dda2bc242a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.151907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:55.260782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ab90d-703d-47ee-bc05-dda2bc242a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:55.262260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ab90d-703d-47ee-bc05-dda2bc242a67 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:55.337671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a037b7-b106-4c36-83c3-4508131162fe None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:55.366528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:55.379438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:55.380294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.381318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.381772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.394730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:55.400885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:55.401959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:55.402924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:55.407523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 172/343] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:45:55.415804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:55.433266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:55.474208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 172/344] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:55.475763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e104c5-c0b6-420a-b5f2-a51d60bb9e95 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.476195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e104c5-c0b6-420a-b5f2-a51d60bb9e95 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.476614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e104c5-c0b6-420a-b5f2-a51d60bb9e95 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.478072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 173/345] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:55.483120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e104c5-c0b6-420a-b5f2-a51d60bb9e95 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:55.553933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5622ee16-a651-477c-a57e-f11acf7abaaa None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:55.595788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc864f-c57f-4a03-bd52-9c61b9266df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:55.597161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc864f-c57f-4a03-bd52-9c61b9266df6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.597591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc864f-c57f-4a03-bd52-9c61b9266df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.598045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc864f-c57f-4a03-bd52-9c61b9266df6 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.599321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc864f-c57f-4a03-bd52-9c61b9266df6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:55.599802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc864f-c57f-4a03-bd52-9c61b9266df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:55.602102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc864f-c57f-4a03-bd52-9c61b9266df6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:55.613844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 173/346] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:55.616675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74f39e-3fdb-4483-b21a-6bb01205d72b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.617111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74f39e-3fdb-4483-b21a-6bb01205d72b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.617529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74f39e-3fdb-4483-b21a-6bb01205d72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.628879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e104c5-c0b6-420a-b5f2-a51d60bb9e95 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:55.653879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4a74f39e-3fdb-4483-b21a-6bb01205d72b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:55.656010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4a74f39e-3fdb-4483-b21a-6bb01205d72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:55.673845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:55.674900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.675523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.675859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.677243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:55.679813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:55.681916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 174/347] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:55 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:55.682870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:55.687421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:55.749146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 174/348] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:55.753726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74f39e-3fdb-4483-b21a-6bb01205d72b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:55.754310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:55.845172 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20235dca-0808-4644-98bc-0eb362ca2cce None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:55.886330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:55.887319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.887802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.888272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.900260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74f39e-3fdb-4483-b21a-6bb01205d72b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:55.901143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:55.906205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:55.907205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 12:45:55.908020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:55.913094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 175/349] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:55.914211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:55.944828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8462a22-3fe6-4038-af8a-48dc0265adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:55.945897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462a22-3fe6-4038-af8a-48dc0265adc3 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:55.946439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462a22-3fe6-4038-af8a-48dc0265adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:55.946881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462a22-3fe6-4038-af8a-48dc0265adc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:55.986993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:56.073526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8462a22-3fe6-4038-af8a-48dc0265adc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:56.075471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8462a22-3fe6-4038-af8a-48dc0265adc3 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:56.086606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86c50a-1275-4640-9493-fbc420d0dbe1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:56.113869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:56.163723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:56.164669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.165160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.165492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.175403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:56.177166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:56.179148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 176/350] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:45:55 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:56.179740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:56.234363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 175/351] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:56.236097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f9c10-51ec-471b-929b-3736ddf4d485 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.236585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f9c10-51ec-471b-929b-3736ddf4d485 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.237157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f9c10-51ec-471b-929b-3736ddf4d485 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.238666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 176/352] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:56.239998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f9c10-51ec-471b-929b-3736ddf4d485 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:56.296772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a568634-7f25-4fa1-bd69-a31497794f06 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:56.317138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:56.335990 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:56.336605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.336891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.337163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.337875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:56.338432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:56.339794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:56.348613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 177/353] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:56.349247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:56.366346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f9c10-51ec-471b-929b-3736ddf4d485 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:56.400421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:56.409903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a44120-4de9-49c8-bf54-4dd981a22599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:56.410553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a44120-4de9-49c8-bf54-4dd981a22599 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.410835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a44120-4de9-49c8-bf54-4dd981a22599 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.411107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a44120-4de9-49c8-bf54-4dd981a22599 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.425543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a44120-4de9-49c8-bf54-4dd981a22599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b5a44120-4de9-49c8-bf54-4dd981a22599 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:56.431006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:56.431877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:56.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 177/354] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:56.433343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1d040-8b70-4c48-b429-0eb04c3feee4 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.433914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1d040-8b70-4c48-b429-0eb04c3feee4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.433914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1d040-8b70-4c48-b429-0eb04c3feee4 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.436077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 178/355] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:56.436077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1d040-8b70-4c48-b429-0eb04c3feee4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:56.476842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ff2cc-9602-430e-8ffc-50cc78aad69c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:56.520303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150cc6e7-2854-4c28-8847-4d8c7316660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:56.520972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cc6e7-2854-4c28-8847-4d8c7316660d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.521257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cc6e7-2854-4c28-8847-4d8c7316660d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.521540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cc6e7-2854-4c28-8847-4d8c7316660d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.522559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150cc6e7-2854-4c28-8847-4d8c7316660d None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=7a1d4145c5484035a6842313c2648cc1) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:56.522875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150cc6e7-2854-4c28-8847-4d8c7316660d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:56.524313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150cc6e7-2854-4c28-8847-4d8c7316660d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:56.533290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 178/356] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:56.535335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370ccdd-9713-4ef4-b564-60e8963f68be None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.535752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370ccdd-9713-4ef4-b564-60e8963f68be None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.536026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370ccdd-9713-4ef4-b564-60e8963f68be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.563046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1d040-8b70-4c48-b429-0eb04c3feee4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:56.567760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8370ccdd-9713-4ef4-b564-60e8963f68be None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:56.569406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8370ccdd-9713-4ef4-b564-60e8963f68be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:56.589255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:56.611085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:56.612497 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.613232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.613512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.615091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:56.617291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:56.619073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 179/357] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:56.619979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:56.667700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 179/358] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:56.669039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370ccdd-9713-4ef4-b564-60e8963f68be None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:56.713289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:56.770588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d797102-2030-4967-8d8d-d0d3007494fc None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:56.804664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370ccdd-9713-4ef4-b564-60e8963f68be None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:56.807538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8af27-aeeb-469f-a648-712dd3a8471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:56.808168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8af27-aeeb-469f-a648-712dd3a8471e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.808444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8af27-aeeb-469f-a648-712dd3a8471e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.808709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8af27-aeeb-469f-a648-712dd3a8471e None admin] PATH_INFO: `/v3/users/eef469e465ab4783b35e1be0d9dd8238` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.809926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8af27-aeeb-469f-a648-712dd3a8471e None admin] RBAC: Authorizing `identity:get_user(user_id=eef469e465ab4783b35e1be0d9dd8238)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:56.811366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8af27-aeeb-469f-a648-712dd3a8471e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:56.813349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 180/359] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/users/eef469e465ab4783b35e1be0d9dd8238 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:56.814504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3e78-b4b4-4551-bd40-cf103cfa4caf None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.814782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3e78-b4b4-4551-bd40-cf103cfa4caf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.815051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3e78-b4b4-4551-bd40-cf103cfa4caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.837537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5a7e3e78-b4b4-4551-bd40-cf103cfa4caf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:56.838887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5a7e3e78-b4b4-4551-bd40-cf103cfa4caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:56.852906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:56.854381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:56.854728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:56.855202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:56.856055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:56.856540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:56.858320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:56.870854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 180/360] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:56.872149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:56.903834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:56.924223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 181/361] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:56.925361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e3e78-b4b4-4551-bd40-cf103cfa4caf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.009974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:57.019298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603c40-afc9-4d61-8190-0492ae6c4775 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.065614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e3e78-b4b4-4551-bd40-cf103cfa4caf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.065835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.066563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.067064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.067734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.074693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:57.077170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:57.079858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 181/362] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:57.080910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.110973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.111877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.112298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.112848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.113891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] RBAC: Adding query filter params (role.id=8f099e1c603944dfb66a49bef8b0b790, user.id=eef469e465ab4783b35e1be0d9dd8238, scope.system=all) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:57.114400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:57.116458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:57.130454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 182/363] 10.222.0.72 () {56 vars in 1308 bytes} [Tue Sep 13 12:45:56 2022] GET /identity/v3/role_assignments?user.id=eef469e465ab4783b35e1be0d9dd8238&scope.system=all&role.id=8f099e1c603944dfb66a49bef8b0b790 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:57.131764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.220198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2febc-3ccc-4dac-aa47-9f610f1746f0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.259596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:57.274499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.274499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.274499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.274499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.314397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:57.339181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45a1a8-b83e-41db-ae72-8d608cb5d66c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.368317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:57.369879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:57.401323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.402097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.402429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.402705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.419583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:57.430443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:57.432975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:45:57.434719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:57.436873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 183/364] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:57.438093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.526660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 182/365] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:57.527435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.564322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb809c-e54b-41a4-bf46-7f649d06fb98 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.566672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:57.601237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:57.613472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.614236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.614576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.614928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.616300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=7a1d4145c5484035a6842313c2648cc1) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:57.616685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:57.618291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:57.627563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 184/366] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:57.631931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.659401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34f794-b0a1-41df-8801-ba25fc2821e9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.711951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.714964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.714964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.715212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.728204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:57.734328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:57.735330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:45:57.736225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 183/367] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:57.736904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.774241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6999189-9a08-4ac7-a54b-f23bb8d320fe None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.823371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.824542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.824823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.825097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.825797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:57.826108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:57.828245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:57.840092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 185/368] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:57.840936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.877677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:57.880800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a65ff-6406-497f-925e-bb1af838deaf None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:57.910002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:57.931460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:57.932114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:57.932396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:57.932669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:57.934599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:57.936140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:57.938324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 184/369] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:57.939119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:57.967232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133c22d-daa6-400e-9bbd-abb11204c70c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.008231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.009271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.009821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.010273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.011133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:58.011651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.013356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.023576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 186/370] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:58.024700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.063781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152bf5c-844c-449b-90ae-77fba3961fbe None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.105041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.105693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.106094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.106445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.149832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:58.162346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f446-2f41-47c8-b371-46adb98e57bb None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.177303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:58.188770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.190339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.202785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.203726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.204246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.204715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.206183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.207960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.210362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 187/371] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:58.211438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.294856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 185/372] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:58.295540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.329216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feebc5c-f442-4bdc-bb63-274f215fe804 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.369118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.370122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.370644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.371207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.388382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:58.398618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cd2e3-5a0d-478c-b260-0c317fbfb857 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.403750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:58.433348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.433984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.434262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.434803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.435264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:58.435569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.436996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.444585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 186/373] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 260 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:58.445138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.450560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.458967 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.556903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135f19a-e3d7-456a-a454-93593c706629 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.566592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 188/374] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:45:58.574619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.602457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.603111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.603392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.603663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.605025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.606368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.608004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 187/375] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:58.613340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.647805 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:58.650582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:58.697456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27948a-881d-4b26-b1be-7b0459fec4fe None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.753104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc218a1-97c7-43f8-a229-cdf227a832b5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:58.758397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.760236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.761788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.766103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] PATH_INFO: `/v3/projects/cdd5b4a37f1d439babc288358ccc8191` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.767727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] RBAC: Authorizing `identity:get_project(project_id=cdd5b4a37f1d439babc288358ccc8191)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.769358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.773862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 189/376] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/projects/cdd5b4a37f1d439babc288358ccc8191 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:58.781227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.813730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:58.814411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:58.814688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:58.814959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:58.816147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:58.817545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:58.819693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 188/377] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:58.827201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:58.949904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:58.962044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7067e0e-6725-42f9-9579-3cdcc4f09f93 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:59.001909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:59.002652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:59.002934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:59.003371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:59.004188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:59.004499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:59.007321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:59.013921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 189/378] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:59.024088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e517ce-a0c1-449e-9718-964b02b4b46c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:59.071328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:59.081139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ede8c-9ca1-4c67-a1ab-1a6dbe560fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:59.083656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ede8c-9ca1-4c67-a1ab-1a6dbe560fdf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:59.083656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ede8c-9ca1-4c67-a1ab-1a6dbe560fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:59.083656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ede8c-9ca1-4c67-a1ab-1a6dbe560fdf None admin] PATH_INFO: `/v3/groups/e2cda8cc63814a068a1eb9c44e9e9d05` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:59.092234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ede8c-9ca1-4c67-a1ab-1a6dbe560fdf None admin] RBAC: Authorizing `identity:get_group(group_id=e2cda8cc63814a068a1eb9c44e9e9d05)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:59.093769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ede8c-9ca1-4c67-a1ab-1a6dbe560fdf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:59.095772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 190/379] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:45:58 2022] GET /identity/v3/groups/e2cda8cc63814a068a1eb9c44e9e9d05 => generated 229 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:45:59.102541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:59.226490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b93db-21f8-4896-b6a9-546700d98b73 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:59.270992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b7224-95dc-48ea-b476-282e660e31fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:59.272544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7224-95dc-48ea-b476-282e660e31fc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:59.273040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7224-95dc-48ea-b476-282e660e31fc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:59.273538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7224-95dc-48ea-b476-282e660e31fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:59.274522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b7224-95dc-48ea-b476-282e660e31fc None admin] RBAC: Adding query filter params (group.id=e2cda8cc63814a068a1eb9c44e9e9d05, role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=cdd5b4a37f1d439babc288358ccc8191) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:45:59.275051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b7224-95dc-48ea-b476-282e660e31fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:45:59.276731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b7224-95dc-48ea-b476-282e660e31fc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:45:59.283883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 190/380] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:45:59 2022] GET /identity/v3/role_assignments?group.id=e2cda8cc63814a068a1eb9c44e9e9d05&scope.project.id=cdd5b4a37f1d439babc288358ccc8191&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:45:59.360984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562292-f140-42a9-8320-c086cd3d0268 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:59.361338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562292-f140-42a9-8320-c086cd3d0268 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:59.361529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba562292-f140-42a9-8320-c086cd3d0268 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:59.363260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 191/381] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:45:59.373439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4b5b8-0b61-4f68-b62a-b8af28f4beae None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:59.374120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4b5b8-0b61-4f68-b62a-b8af28f4beae None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:59.374610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4b5b8-0b61-4f68-b62a-b8af28f4beae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:59.401870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-43a4b5b8-0b61-4f68-b62a-b8af28f4beae None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:45:59.403447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-43a4b5b8-0b61-4f68-b62a-b8af28f4beae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:45:59.419466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:45:59.491915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 191/382] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:45:59.806218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562292-f140-42a9-8320-c086cd3d0268 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:45:59.927316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba562292-f140-42a9-8320-c086cd3d0268 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:45:59.968068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:45:59.968749 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:45:59.969022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:45:59.969291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:45:59.984066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:45:59.989058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:45:59.989972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:45:59.991318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 192/383] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:45:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:45:59.996854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4b5b8-0b61-4f68-b62a-b8af28f4beae None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:00.107154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4b5b8-0b61-4f68-b62a-b8af28f4beae None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:00.148355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab8ffa-9e30-48bd-805b-1cdaa1b82e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:00.149027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8ffa-9e30-48bd-805b-1cdaa1b82e52 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:00.149308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8ffa-9e30-48bd-805b-1cdaa1b82e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:00.149575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab8ffa-9e30-48bd-805b-1cdaa1b82e52 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:00.150268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab8ffa-9e30-48bd-805b-1cdaa1b82e52 None admin] RBAC: Adding query filter params (name=member) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:00.150567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab8ffa-9e30-48bd-805b-1cdaa1b82e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:00.151921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab8ffa-9e30-48bd-805b-1cdaa1b82e52 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:00.166322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 192/384] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:45:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:00.172109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:00.249874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:00.309506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a2c7b-0c95-48d3-a9d1-1b6c7aba8350 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:00.337429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a1fa2-6d1d-43eb-8364-8a39ca3028e2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:00.337758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a1fa2-6d1d-43eb-8364-8a39ca3028e2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:00.338040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a1fa2-6d1d-43eb-8364-8a39ca3028e2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:00.339371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 193/385] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:46:00.346001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b024092-83f7-4cc5-b604-46c821ac0b55 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:00.346273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b024092-83f7-4cc5-b604-46c821ac0b55 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:00.346548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b024092-83f7-4cc5-b604-46c821ac0b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:00.372162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:00.372721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:00.372995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:00.373265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:00.374531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0b024092-83f7-4cc5-b604-46c821ac0b55 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:00.375713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0b024092-83f7-4cc5-b604-46c821ac0b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:00.378259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:00.379830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:00.382309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 193/386] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:00 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:00.387957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:00.390083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:00.512935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 194/387] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:00.562225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f6b00-3775-4f4b-b19c-a93c51a85aba None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:00.589231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:00.608932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:00.609575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:00.609917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:00.610190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:00.614425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:00.616502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:00.618351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 194/388] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:00 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:00.624397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b024092-83f7-4cc5-b604-46c821ac0b55 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:00.757645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b024092-83f7-4cc5-b604-46c821ac0b55 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:00.797701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:00.798399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:00.798680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:00.798950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:00.800829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a1d4145c5484035a6842313c2648cc1, group_id=fcb1d6c867d04a92ba4d4b92feb90874, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:00.802314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:00.882703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:00.887197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 195/389] 10.222.0.72 () {58 vars in 1442 bytes} [Tue Sep 13 12:46:00 2022] PUT /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:00.982055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:01.132001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104592b6-1b53-4a3b-9675-f06c58762f86 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:01.173231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6307f-b079-4ee7-a363-0734ed3354f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:01.173231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6307f-b079-4ee7-a363-0734ed3354f7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:01.173231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6307f-b079-4ee7-a363-0734ed3354f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:01.173231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6307f-b079-4ee7-a363-0734ed3354f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:01.182363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6307f-b079-4ee7-a363-0734ed3354f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-adc6307f-b079-4ee7-a363-0734ed3354f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:01.188071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:01.188960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:01.189777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:01.192112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 195/390] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:01.198450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:01.215171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce962b-cd89-43b2-8e46-d77625667acc None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:01.215461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce962b-cd89-43b2-8e46-d77625667acc None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:01.215726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce962b-cd89-43b2-8e46-d77625667acc None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:01.217207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 196/391] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:01.222152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5184-f044-4af4-9c5e-959b213d69ff None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:01.222439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5184-f044-4af4-9c5e-959b213d69ff None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:01.222709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd5184-f044-4af4-9c5e-959b213d69ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:01.247886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0abd5184-f044-4af4-9c5e-959b213d69ff None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:01.249481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0abd5184-f044-4af4-9c5e-959b213d69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:01.332474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:01.380023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4bba7-db9e-4cf3-aaf4-4a48dbf701f3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:01.384391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 197/392] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:46:01.417424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:01.418416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:01.418896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:01.419383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:01.420258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:01.420773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:01.422411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:01.432331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 196/393] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:01.438352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5184-f044-4af4-9c5e-959b213d69ff None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:01.576215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd5184-f044-4af4-9c5e-959b213d69ff None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:01.627179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:01.627813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:01.628089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:01.628360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:01.629464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:01.632048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:01.634119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 198/394] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:01 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:01.646261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:01.711355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:01.766134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:01.786754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:01.844554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe5e2-2106-4ee2-bf68-5b86ee887780 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:01.886894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4117e3-139d-445b-ae52-5e7a75e36765 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:01.901867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:01.902510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:01.902791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:01.903065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:01.904201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:01.905642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:01.907601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 197/395] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:01 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:01.913626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:02.016871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:02.017430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.017704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.017996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.040852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:02.045782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:02.046735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:46:02.047551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:02.048378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 199/396] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:02.058072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:02.130104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8919f3-07f7-4f19-978f-bab586045019 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:02.167201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:02.175493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d536e-82ec-4d50-9f88-427155b79f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:02.181215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d536e-82ec-4d50-9f88-427155b79f70 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.181215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d536e-82ec-4d50-9f88-427155b79f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.181215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d536e-82ec-4d50-9f88-427155b79f70 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.181215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d536e-82ec-4d50-9f88-427155b79f70 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a1d4145c5484035a6842313c2648cc1, group_id=fcb1d6c867d04a92ba4d4b92feb90874, role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:02.182131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d536e-82ec-4d50-9f88-427155b79f70 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:02.193002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb08f4-eb6b-4bc2-95a4-c80a7fd6ea8d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:02.202024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:02.229723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79cb24-4a3a-4bda-8f21-f08ac9543958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:02.230368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79cb24-4a3a-4bda-8f21-f08ac9543958 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.230650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79cb24-4a3a-4bda-8f21-f08ac9543958 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.230925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79cb24-4a3a-4bda-8f21-f08ac9543958 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.231591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79cb24-4a3a-4bda-8f21-f08ac9543958 None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:02.234670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79cb24-4a3a-4bda-8f21-f08ac9543958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:02.234670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79cb24-4a3a-4bda-8f21-f08ac9543958 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:02.242230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 200/397] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:46:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:02.243515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1912cc-db16-4670-9478-ba01d969020b None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.243804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1912cc-db16-4670-9478-ba01d969020b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.244076 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1912cc-db16-4670-9478-ba01d969020b None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.245419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 201/398] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:02.246380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caf0c5-5674-4eef-93e9-9a57d8ac1666 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.246800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caf0c5-5674-4eef-93e9-9a57d8ac1666 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.247020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caf0c5-5674-4eef-93e9-9a57d8ac1666 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.248384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 202/399] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:02.253251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1caf0c5-5674-4eef-93e9-9a57d8ac1666 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:02.263886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 198/400] 10.222.0.72 () {58 vars in 1442 bytes} [Tue Sep 13 12:46:01 2022] PUT /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:02.263886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0565a-0166-43c6-910f-ca88e33d51e8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.263886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0565a-0166-43c6-910f-ca88e33d51e8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.263886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0565a-0166-43c6-910f-ca88e33d51e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.290190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-90d0565a-0166-43c6-910f-ca88e33d51e8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:02.290190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-90d0565a-0166-43c6-910f-ca88e33d51e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:02.379656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1caf0c5-5674-4eef-93e9-9a57d8ac1666 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:02.389245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 199/401] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:02.392692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73361d-52fb-4aae-871f-6feb5a4e4c70 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.393199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73361d-52fb-4aae-871f-6feb5a4e4c70 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.394335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73361d-52fb-4aae-871f-6feb5a4e4c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.435496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8d73361d-52fb-4aae-871f-6feb5a4e4c70 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:02.438169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8d73361d-52fb-4aae-871f-6feb5a4e4c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:02.456595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:02.457104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.457378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.457645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.460754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:02.460964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:02.463994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 203/402] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:02 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:02.469150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:02.505573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:02.528870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 200/403] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:02.581542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4eaeb-0cb5-49b4-9189-02fde9713afc None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:02.625105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:02.626009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.626430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.626843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.630413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:02.632778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:02.634967 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 204/404] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:02 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:02.641259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73361d-52fb-4aae-871f-6feb5a4e4c70 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:02.701472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:02.723954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:02.817379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beebb6c-bee8-4e9d-a284-fc635dc175ef None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:02.820027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:02.828143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73361d-52fb-4aae-871f-6feb5a4e4c70 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:02.862435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:02.863001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.863282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.863555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.877012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:02.882060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:02.882200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:02.883050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:02.883910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:02.884522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:02.884522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:02.884522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:02.884690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 205/405] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:02.884690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:02.891017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] RBAC: Authorizing `identity:create_grant(project_id=e1a6b26bf495458796a0a05b1b3d03ea, group_id=fcb1d6c867d04a92ba4d4b92feb90874, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:02.892534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:02.993697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04693b-1475-4e45-a7c5-c5495f5c4a73 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.033785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.034469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.034745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.035020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.035851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 201/406] 10.222.0.72 () {58 vars in 1442 bytes} [Tue Sep 13 12:46:02 2022] PUT /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:03.036902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.053592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:03.062990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:03.063970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:03.064763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 206/407] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:03.069543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.101705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:03.191486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0651a3-67f5-4f64-862a-d21f2af76d8d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.231766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.231766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.231766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.232545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.232826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:03.232985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:03.234448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:03.244412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 207/408] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:03.250486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.260348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb240b-9e0f-42c7-9b5e-27db468bc76a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.280984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:03.310103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.311065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.311670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.312140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.313024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:03.313539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:03.316826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:03.336889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 202/409] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:03.344204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.409784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:03.429254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117c557-62bd-4260-a865-c963fc9fa24b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.482829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.483507 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.483788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.484065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.490012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:03.492595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:03.500324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 208/410] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:03 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:03.507206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.531820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97b1fe-c14a-4d1d-b81c-84c0558b6c1d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.610023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6d231-b433-4482-abea-76cfb86c008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.610675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d231-b433-4482-abea-76cfb86c008c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.610957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d231-b433-4482-abea-76cfb86c008c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.611237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d231-b433-4482-abea-76cfb86c008c None admin] PATH_INFO: `/v3/groups/e2cda8cc63814a068a1eb9c44e9e9d05` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.613380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6d231-b433-4482-abea-76cfb86c008c None admin] RBAC: Authorizing `identity:get_group(group_id=e2cda8cc63814a068a1eb9c44e9e9d05)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:03.614857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6d231-b433-4482-abea-76cfb86c008c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:03.616485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 203/411] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:03 2022] GET /identity/v3/groups/e2cda8cc63814a068a1eb9c44e9e9d05 => generated 229 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:03.617629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f0c3-1275-4e3c-ba74-053b78a1e461 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.617938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f0c3-1275-4e3c-ba74-053b78a1e461 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.618216 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f0c3-1275-4e3c-ba74-053b78a1e461 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.619441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 204/412] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:03.623851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35f0c3-1275-4e3c-ba74-053b78a1e461 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.675007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:03.695051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e145634-49f0-41d7-8117-36c2946fdf2b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.741830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5d545-85e7-44f0-9482-1b47436f29ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.742774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5d545-85e7-44f0-9482-1b47436f29ed None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.743216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5d545-85e7-44f0-9482-1b47436f29ed None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.743760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5d545-85e7-44f0-9482-1b47436f29ed None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.744647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35f0c3-1275-4e3c-ba74-053b78a1e461 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.745424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5d545-85e7-44f0-9482-1b47436f29ed None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:03.748210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5d545-85e7-44f0-9482-1b47436f29ed None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:03.750108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 209/413] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:03 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:03.751471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8143ea0-578b-4ce3-8f47-0040873c923b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.751938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8143ea0-578b-4ce3-8f47-0040873c923b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.752937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8143ea0-578b-4ce3-8f47-0040873c923b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.772636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d8143ea0-578b-4ce3-8f47-0040873c923b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:03.774144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d8143ea0-578b-4ce3-8f47-0040873c923b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:03.783639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.784580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.784833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.785174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] PATH_INFO: `/v3/projects/cdd5b4a37f1d439babc288358ccc8191` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.786613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] RBAC: Authorizing `identity:get_project(project_id=cdd5b4a37f1d439babc288358ccc8191)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:03.788836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:03.790998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 205/414] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:03 2022] GET /identity/v3/projects/cdd5b4a37f1d439babc288358ccc8191 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:03.791863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.799400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:03.857403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 210/415] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:03.859014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8143ea0-578b-4ce3-8f47-0040873c923b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:03.907520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fba1b-6f16-4bb3-acd7-f90d6801673f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:03.916717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:03.970571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:03.971236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:03.971517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:03.971792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:03.974017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1a6b26bf495458796a0a05b1b3d03ea, group_id=fcb1d6c867d04a92ba4d4b92feb90874, role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:03.976406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:03.995062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8143ea0-578b-4ce3-8f47-0040873c923b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:04.027927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:04.028602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:04.028866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:04.029136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] PATH_INFO: `/v3/projects/cdd5b4a37f1d439babc288358ccc8191/groups/e2cda8cc63814a068a1eb9c44e9e9d05/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:04.031163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdd5b4a37f1d439babc288358ccc8191, group_id=e2cda8cc63814a068a1eb9c44e9e9d05, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:04.032619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:04.061948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 206/416] 10.222.0.72 () {58 vars in 1442 bytes} [Tue Sep 13 12:46:03 2022] PUT /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea/groups/fcb1d6c867d04a92ba4d4b92feb90874/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:04.135570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 211/417] 10.222.0.72 () {58 vars in 1442 bytes} [Tue Sep 13 12:46:03 2022] PUT /identity/v3/projects/cdd5b4a37f1d439babc288358ccc8191/groups/e2cda8cc63814a068a1eb9c44e9e9d05/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:04.258482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:04.387759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dd91f-dfc5-4661-af30-d81f353ab820 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:04.424165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:04.425110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:04.425561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:04.426047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:04.437231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:04.442792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:04.444348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:46:04.445134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:04.447380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 207/418] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:04.459422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:04.480487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:04.598360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72575e0b-989e-46ac-bf36-ed5e2675dda8 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:04.653260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a51e09-aa6a-49d2-b4bc-7cbbce3664c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:04.653955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a51e09-aa6a-49d2-b4bc-7cbbce3664c1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:04.654229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a51e09-aa6a-49d2-b4bc-7cbbce3664c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:04.654543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a51e09-aa6a-49d2-b4bc-7cbbce3664c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:04.655221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a51e09-aa6a-49d2-b4bc-7cbbce3664c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:04.655517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a51e09-aa6a-49d2-b4bc-7cbbce3664c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:04.656881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a51e09-aa6a-49d2-b4bc-7cbbce3664c1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:04.672533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:04.673597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 212/419] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:04.729230 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:04.830777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0f7d6-cba6-4fc3-a3a4-8cc54f21da7b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:04.879886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:04.880427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:04.881215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:04.881579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:04.969142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:04.970650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:04.990896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cc789-8dc8-473a-ac78-684836a7cc44 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:04.991199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cc789-8dc8-473a-ac78-684836a7cc44 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:04.991447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cc789-8dc8-473a-ac78-684836a7cc44 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:04.993076 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 213/420] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:05.005494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d36560-36d6-45e5-aa25-08825fdbd9a4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.006362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d36560-36d6-45e5-aa25-08825fdbd9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.006650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d36560-36d6-45e5-aa25-08825fdbd9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.009909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:05.032399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-95d36560-36d6-45e5-aa25-08825fdbd9a4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:05.033607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-95d36560-36d6-45e5-aa25-08825fdbd9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:05.123224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 214/421] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:46:05.142776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 208/422] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:05.154544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d36560-36d6-45e5-aa25-08825fdbd9a4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:05.159023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:05.297055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d36560-36d6-45e5-aa25-08825fdbd9a4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:05.341268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:05.342022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.342306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.342584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.346414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:05.347871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:05.349896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 215/423] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:05 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:05.355133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:05.452036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:05.508545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c19ee7-2a29-4e52-8bd5-b7e2630c9ad9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:05.550509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:05.560750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:05.561887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.562179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.571525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.571525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:05.571525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:05.571525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 209/424] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:05 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:05.576979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:05.591529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:05.654061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aea8c-8bd8-412a-ab34-40d3be5558d3 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:05.718846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe89332-5183-497d-8533-74e2a81568b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:05.719407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe89332-5183-497d-8533-74e2a81568b9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.719684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe89332-5183-497d-8533-74e2a81568b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.719955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe89332-5183-497d-8533-74e2a81568b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.733012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe89332-5183-497d-8533-74e2a81568b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-0fe89332-5183-497d-8533-74e2a81568b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:05.740937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:05.742005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:05.742789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:05.745582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 216/425] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:05.748336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7feb81-228a-46d3-9e14-c749a5a3bf18 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.749143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7feb81-228a-46d3-9e14-c749a5a3bf18 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.749981 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7feb81-228a-46d3-9e14-c749a5a3bf18 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.752158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 217/426] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:05.753159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7feb81-228a-46d3-9e14-c749a5a3bf18 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:05.758820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62aaf07-8412-427a-8e74-ed00c109b434 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:05.793919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a67c03-d5fb-4565-a102-e7b48539f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:05.794670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a67c03-d5fb-4565-a102-e7b48539f2dc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.795049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a67c03-d5fb-4565-a102-e7b48539f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.795390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a67c03-d5fb-4565-a102-e7b48539f2dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.796189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a67c03-d5fb-4565-a102-e7b48539f2dc None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=7a1d4145c5484035a6842313c2648cc1) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:05.796567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a67c03-d5fb-4565-a102-e7b48539f2dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:05.798095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a67c03-d5fb-4565-a102-e7b48539f2dc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:05.809899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 210/427] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:46:05 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:05.812321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9ae9-db45-4f97-85e9-4040d75b9c3b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.812321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9ae9-db45-4f97-85e9-4040d75b9c3b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.812321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9ae9-db45-4f97-85e9-4040d75b9c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.847023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1f9f9ae9-db45-4f97-85e9-4040d75b9c3b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:05.848338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1f9f9ae9-db45-4f97-85e9-4040d75b9c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:05.882632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7feb81-228a-46d3-9e14-c749a5a3bf18 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:05.909543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:05.921831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 211/428] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:05.942139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:05.950111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:05.950996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:05.951715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:05.952148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:05.953020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:05.953474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:05.955121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:05.970427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 218/429] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:05.976708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f9ae9-db45-4f97-85e9-4040d75b9c3b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:06.138499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f9ae9-db45-4f97-85e9-4040d75b9c3b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:06.174235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:06.174844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.175123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.175394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.237090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:06.242868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:06.253095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:06.260408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:06.368435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601ce2-f230-477f-8dd7-2e2c9f0198d3 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:06.377763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 212/430] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:06.382066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:06.429706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:06.430353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.430635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.430910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.453287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:06.461760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:06.462688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 12:46:06.463465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:06.466535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 219/431] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:06.473062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:06.527185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:06.538878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091de4f-f5eb-4751-b5a5-293ca92388cb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:06.542659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:06.581103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:06.582198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.582785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.583259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] PATH_INFO: `/v3/projects/7a1d4145c5484035a6842313c2648cc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.585392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7a1d4145c5484035a6842313c2648cc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:06.587179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:06.590764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 213/432] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:06 2022] GET /identity/v3/projects/7a1d4145c5484035a6842313c2648cc1 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:06.597244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:06.607773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73f496-c2dd-43e1-940a-fd7bafbe015f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:06.653779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:06.653779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.653779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.653779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.654227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] RBAC: Adding query filter params (name=member) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:06.654227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:06.655193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:06.665435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 220/433] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:46:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:06.674165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:06.730684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505e7a3-f30e-4829-b311-20f7d1a054e4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:06.772698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929534-33a2-45d0-b25c-48be83631356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:06.773333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929534-33a2-45d0-b25c-48be83631356 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.773613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929534-33a2-45d0-b25c-48be83631356 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.774018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929534-33a2-45d0-b25c-48be83631356 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.775193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34929534-33a2-45d0-b25c-48be83631356 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:06.776716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34929534-33a2-45d0-b25c-48be83631356 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:06.778672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 214/434] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:06 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:06.779645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de82e6-7997-4834-b265-6816dfc01a80 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.779936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de82e6-7997-4834-b265-6816dfc01a80 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.780192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de82e6-7997-4834-b265-6816dfc01a80 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.790232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 215/435] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:06.791585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de82e6-7997-4834-b265-6816dfc01a80 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:06.799997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2016272-38fc-4d9f-b743-8f36618f406b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:06.816301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:06.825147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:06.837270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14cabe-72ab-43b3-a43c-1de1472890e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:06.838183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cabe-72ab-43b3-a43c-1de1472890e0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.838657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cabe-72ab-43b3-a43c-1de1472890e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.839119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cabe-72ab-43b3-a43c-1de1472890e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.907178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de82e6-7997-4834-b265-6816dfc01a80 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:06.914241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14cabe-72ab-43b3-a43c-1de1472890e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:06.915740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14cabe-72ab-43b3-a43c-1de1472890e0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:06.945179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c1a19-de94-4d2e-a553-d888845d900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:06.945802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c1a19-de94-4d2e-a553-d888845d900e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.946128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c1a19-de94-4d2e-a553-d888845d900e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.946399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c1a19-de94-4d2e-a553-d888845d900e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.947130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c1a19-de94-4d2e-a553-d888845d900e None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=7a1d4145c5484035a6842313c2648cc1) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:06.947435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c1a19-de94-4d2e-a553-d888845d900e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:06.949134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c1a19-de94-4d2e-a553-d888845d900e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:06.962178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 216/436] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:46:06 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=7a1d4145c5484035a6842313c2648cc1&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:06.963604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfd965-a3f0-403f-b7b7-5fc858075ee9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:06.963998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfd965-a3f0-403f-b7b7-5fc858075ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:06.964207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfd965-a3f0-403f-b7b7-5fc858075ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:06.993594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c9bfd965-a3f0-403f-b7b7-5fc858075ee9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:06.995122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c9bfd965-a3f0-403f-b7b7-5fc858075ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:07.027272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 221/437] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 12:46:07.028552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539a42d-ba99-4597-b95d-556a43bfaa75 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.028897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539a42d-ba99-4597-b95d-556a43bfaa75 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.029162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539a42d-ba99-4597-b95d-556a43bfaa75 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.030346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 222/438] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:07.032419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539a42d-ba99-4597-b95d-556a43bfaa75 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.051674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:07.084907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 217/439] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:07.086325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9947fa-ddfb-4b1c-8db0-ada0c8ddbc6c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.086615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9947fa-ddfb-4b1c-8db0-ada0c8ddbc6c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.086885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9947fa-ddfb-4b1c-8db0-ada0c8ddbc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.119689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8a9947fa-ddfb-4b1c-8db0-ada0c8ddbc6c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:07.121312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8a9947fa-ddfb-4b1c-8db0-ada0c8ddbc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:07.141528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:07.160010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539a42d-ba99-4597-b95d-556a43bfaa75 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.204699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.205900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.206378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.206821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.208451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:07.210049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:07.211718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 218/440] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:07.212001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 223/441] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:07.218576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9947fa-ddfb-4b1c-8db0-ada0c8ddbc6c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.321028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9947fa-ddfb-4b1c-8db0-ada0c8ddbc6c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.357349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.358522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.359770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.360053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.360328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.361486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:07.363499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:07.364873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 219/442] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:07.370926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.394834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:07.467087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:07.485365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387e381-7a2f-400b-b3af-894ebac7e892 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.496746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b817b-731e-4361-9c3b-3153e59d3fd8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.519474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.520095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.520373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.520642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.530842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.531318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:07.531442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.531721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.531995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.532720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=4279c61c3b4245e39915c7d4d0293dfe, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:07.533028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:07.534474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:07.535932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:07.536781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 224/443] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:07.538988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.550193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 220/444] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=4279c61c3b4245e39915c7d4d0293dfe => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:07.550193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.637293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:07.659069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b073d9-45f1-4942-ac9e-90977013c107 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.673930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7126c61-43f7-4d93-838b-a0f81148d94f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.702084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.702706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.702989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.703263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.703945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:07.704250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:07.705646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:07.714422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 221/445] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:07.719950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.720117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.720596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.720869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.721138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.734409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:07.739917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:07.741532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:07.742389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:07.743003 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 225/446] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:07.748235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.752494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:07.852556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f706d-4b98-43fb-8284-be39cdac0cf8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.872667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c965aa6-ae99-40ce-85c7-797655ef13ad None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:07.893635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.894199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.894483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.895683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.911708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:07.917385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:07.918063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:07.918344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:07.918618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:07.919283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:07.919614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:07.920973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:07.931908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 226/447] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:07.936919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:07.975275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:07.976741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.002511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:08.037053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a2cd5-e857-4537-97f5-cb163f1da2cd None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:08.071013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:08.071607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:08.071885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:08.072158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:08.072850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 222/448] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:08.078642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:08.122762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:08.139390 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:08.140943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.176227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f054d-a557-49c8-a920-51d7d0eb4afe None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:08.204151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:08.210298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:08.210926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:08.211207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:08.211485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] PATH_INFO: `/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:08.212591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=e1a6b26bf495458796a0a05b1b3d03ea)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:08.214064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.215847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 223/449] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:08 2022] GET /identity/v3/projects/e1a6b26bf495458796a0a05b1b3d03ea => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:08.221173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:08.238785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 227/450] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 12:46:08.245175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:08.320141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd1018-1e3a-4ec5-998e-ec7ed072d6c6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:08.332426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:08.341078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004bf6f-e788-4c18-af47-e8ab8c70dacd None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:08.353753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:08.354391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:08.354674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:08.354944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] PATH_INFO: `/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:08.356064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] RBAC: Authorizing `identity:get_group(group_id=fcb1d6c867d04a92ba4d4b92feb90874)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:08.357528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.359134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 224/451] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:08 2022] GET /identity/v3/groups/fcb1d6c867d04a92ba4d4b92feb90874 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:08.364821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:08.375105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:08.375739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:08.376018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:08.376289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] PATH_INFO: `/v3/projects/cdd5b4a37f1d439babc288358ccc8191` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:08.377416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] RBAC: Authorizing `identity:get_project(project_id=cdd5b4a37f1d439babc288358ccc8191)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:08.378925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.380857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 228/452] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:46:08 2022] GET /identity/v3/projects/cdd5b4a37f1d439babc288358ccc8191 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:08.386118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:08.419794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:08.467200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38d31c-4222-4e24-8b65-a7be6b9fa948 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:08.482436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee061a0b-a747-4ebe-a1a6-c765c39ab23f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:08.502078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:08.502835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:08.503120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:08.503399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:08.504131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] RBAC: Adding query filter params (group.id=fcb1d6c867d04a92ba4d4b92feb90874, role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:08.504451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:08.506496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.512542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 225/453] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:46:08 2022] GET /identity/v3/role_assignments?group.id=fcb1d6c867d04a92ba4d4b92feb90874&scope.project.id=e1a6b26bf495458796a0a05b1b3d03ea&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:08.515027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc80fb5-91fc-4cd2-b8cb-a98a74709210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:08.515651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80fb5-91fc-4cd2-b8cb-a98a74709210 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:08.515927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80fb5-91fc-4cd2-b8cb-a98a74709210 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:08.516196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc80fb5-91fc-4cd2-b8cb-a98a74709210 None admin] PATH_INFO: `/v3/groups/e2cda8cc63814a068a1eb9c44e9e9d05` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:08.517330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc80fb5-91fc-4cd2-b8cb-a98a74709210 None admin] RBAC: Authorizing `identity:get_group(group_id=e2cda8cc63814a068a1eb9c44e9e9d05)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:08.518833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc80fb5-91fc-4cd2-b8cb-a98a74709210 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.520447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 229/454] 10.222.0.72 () {56 vars in 1180 bytes} [Tue Sep 13 12:46:08 2022] GET /identity/v3/groups/e2cda8cc63814a068a1eb9c44e9e9d05 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:08.527530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:08.578361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:08.641009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f83104-f612-441d-87cb-a153febe1696 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:08.679957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2cdc9-59da-4e00-ba08-1a1e3cc9d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:08.680560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2cdc9-59da-4e00-ba08-1a1e3cc9d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:08.680836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2cdc9-59da-4e00-ba08-1a1e3cc9d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:08.681111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2cdc9-59da-4e00-ba08-1a1e3cc9d6d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:08.681872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2cdc9-59da-4e00-ba08-1a1e3cc9d6d5 None admin] RBAC: Adding query filter params (group.id=e2cda8cc63814a068a1eb9c44e9e9d05, role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=cdd5b4a37f1d439babc288358ccc8191) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:08.682181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2cdc9-59da-4e00-ba08-1a1e3cc9d6d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:08.683589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2cdc9-59da-4e00-ba08-1a1e3cc9d6d5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:08.689578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 226/455] 10.222.0.72 () {56 vars in 1376 bytes} [Tue Sep 13 12:46:08 2022] GET /identity/v3/role_assignments?group.id=e2cda8cc63814a068a1eb9c44e9e9d05&scope.project.id=cdd5b4a37f1d439babc288358ccc8191&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:11.421322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6cb1-5001-49f7-8951-b6a2918d83c8 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:11.421866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6cb1-5001-49f7-8951-b6a2918d83c8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:11.421943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6cb1-5001-49f7-8951-b6a2918d83c8 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:11.423622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 230/456] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:11.430212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2334-ac6c-4318-a32b-555c09370586 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:11.430737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2334-ac6c-4318-a32b-555c09370586 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:11.430737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2334-ac6c-4318-a32b-555c09370586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:11.459885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-83cd2334-ac6c-4318-a32b-555c09370586 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:11.461409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-83cd2334-ac6c-4318-a32b-555c09370586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:11.548510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc95c1-722c-449f-8d3e-790a8f41f1a0 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:11.548842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc95c1-722c-449f-8d3e-790a8f41f1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:11.549147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc95c1-722c-449f-8d3e-790a8f41f1a0 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:11.549437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 227/457] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:11.550545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 231/458] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:46:11.567087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543379-128d-498f-a7aa-35bf18a85fcd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:11.567371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543379-128d-498f-a7aa-35bf18a85fcd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:11.567644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543379-128d-498f-a7aa-35bf18a85fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:11.593883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-81543379-128d-498f-a7aa-35bf18a85fcd None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:11.595072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-81543379-128d-498f-a7aa-35bf18a85fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:11.642870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:11.698368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 228/459] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:11.842926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034e62-994a-4f69-9aad-af8c66493b75 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:11.843591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034e62-994a-4f69-9aad-af8c66493b75 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:11.844090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034e62-994a-4f69-9aad-af8c66493b75 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:11.845880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 232/460] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:11.851741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67937d44-29fe-4863-832e-f5e36ffefaa2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:11.852031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67937d44-29fe-4863-832e-f5e36ffefaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:11.852309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67937d44-29fe-4863-832e-f5e36ffefaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:11.874318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-67937d44-29fe-4863-832e-f5e36ffefaa2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:11.875559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-67937d44-29fe-4863-832e-f5e36ffefaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:11.879255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d034e62-994a-4f69-9aad-af8c66493b75 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:11.998969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 229/461] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:12.046607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67937d44-29fe-4863-832e-f5e36ffefaa2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:12.057611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d034e62-994a-4f69-9aad-af8c66493b75 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:12.061183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:12.123381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:12.124384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.124884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.125353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.140333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:12.150187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:12.159921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:12.165015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67937d44-29fe-4863-832e-f5e36ffefaa2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:12.171435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:12.172345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:12.173186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:12.173186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:12.173186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:12.175257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 233/462] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:12.196338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:12.215901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:12.216570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.216852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.217125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.235953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:12.238020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:12.273627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:12.274592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:12.275462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:12.275462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:12.275462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:12.275462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 230/463] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:12.284973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:12.383640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7a4e1-b433-477c-9ca6-e2911f4b7e5c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:12.409374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d19d2d-1a10-4789-b64c-071dd451e87d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:12.419776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:12.420494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.420775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.421049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.421696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:12.422272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:12.423621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:12.437432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 234/464] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:12.438338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:12.449798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c883d8d-3486-4e89-be41-3abdf694577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:12.450398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c883d8d-3486-4e89-be41-3abdf694577a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.450678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c883d8d-3486-4e89-be41-3abdf694577a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.450952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c883d8d-3486-4e89-be41-3abdf694577a None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.451871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c883d8d-3486-4e89-be41-3abdf694577a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:12.452171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c883d8d-3486-4e89-be41-3abdf694577a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:12.453516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c883d8d-3486-4e89-be41-3abdf694577a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:12.457097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:12.465371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 231/465] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:12.473497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffb13f-f4b0-4824-8156-6536ae67ce45 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.473497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffb13f-f4b0-4824-8156-6536ae67ce45 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.473497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccffb13f-f4b0-4824-8156-6536ae67ce45 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.473497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 232/466] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:12.473497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffb13f-f4b0-4824-8156-6536ae67ce45 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:12.486955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:12.554285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d07d0-e404-4114-a69f-c6b7ab12e31d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:12.585553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccffb13f-f4b0-4824-8156-6536ae67ce45 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:12.591920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:12.592715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.593496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.593940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.604475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:12.605958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:12.621457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:12.622394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:12.623223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:12.623223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:12.623223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:12.623223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 235/467] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:12.623438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:12.628773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a911b-fdb0-45ac-b7c4-3b3315f94d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:12.629420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a911b-fdb0-45ac-b7c4-3b3315f94d5a None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.629686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a911b-fdb0-45ac-b7c4-3b3315f94d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.629992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a911b-fdb0-45ac-b7c4-3b3315f94d5a None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.634158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a911b-fdb0-45ac-b7c4-3b3315f94d5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:12.634158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a911b-fdb0-45ac-b7c4-3b3315f94d5a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:12.696344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:12.730363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09d22d-6bea-4776-9ac6-bee2480f305a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:12.736713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 233/468] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:46:12 2022] POST /identity/v3/users => generated 246 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:12.738003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dda52-506e-4c50-aedc-097e40374ce8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.738291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dda52-506e-4c50-aedc-097e40374ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.738565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dda52-506e-4c50-aedc-097e40374ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.764694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-675dda52-506e-4c50-aedc-097e40374ce8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:12.766363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-675dda52-506e-4c50-aedc-097e40374ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:12.785244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d989d07-ce40-4f92-b113-72ac90c0a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:12.786483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989d07-ce40-4f92-b113-72ac90c0a8cf None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.787491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989d07-ce40-4f92-b113-72ac90c0a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.788132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989d07-ce40-4f92-b113-72ac90c0a8cf None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.789229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d989d07-ce40-4f92-b113-72ac90c0a8cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:12.791311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d989d07-ce40-4f92-b113-72ac90c0a8cf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:12.841926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:12.853287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 234/469] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:12.854743 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dda52-506e-4c50-aedc-097e40374ce8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:12.916929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 236/470] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:46:12 2022] POST /identity/v3/users => generated 248 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:12.923702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d989b2-2c18-4ff2-99b6-de8740be2a67 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.924052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d989b2-2c18-4ff2-99b6-de8740be2a67 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.924337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d989b2-2c18-4ff2-99b6-de8740be2a67 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.925977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 237/471] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:12.934272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d466-8aca-45d0-bbac-d700cc428984 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:12.934551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d466-8aca-45d0-bbac-d700cc428984 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:12.934824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d466-8aca-45d0-bbac-d700cc428984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:12.961565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-fea9d466-8aca-45d0-bbac-d700cc428984 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:12.962975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fea9d466-8aca-45d0-bbac-d700cc428984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:12.997974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dda52-506e-4c50-aedc-097e40374ce8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:13.041726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:13.042765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:13.043245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:13.043771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:13.044648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:13.045177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:13.046911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:13.059929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 235/472] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:13.065200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 238/473] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:13.066373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:13.152454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:13.157101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9d466-8aca-45d0-bbac-d700cc428984 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:13.210239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e6d7d-9d29-472a-80ee-fdd97fda9492 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:13.227648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:13.248540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:13.249245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:13.249664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:13.250048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:13.250772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:13.252441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:13.282616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9d466-8aca-45d0-bbac-d700cc428984 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:13.323568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:13.324492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:13.324941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:13.325377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:13.337545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:13.339357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:13.354932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:13.355857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:13.356693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:13.356693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:13.356693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:13.361832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 239/474] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:13.364618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:13.370035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 236/475] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:46:13 2022] POST /identity/v3/users => generated 248 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:13.378956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:13.516931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cad8e-1a21-4538-a9dc-84db8e17bb4d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:13.519638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:13.555332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:13.556104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:13.556390 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:13.556658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:13.557602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:13.558133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:13.559854 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:13.573703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 240/476] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:13.579429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:13.582512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702044e6-cda5-403d-b856-d4729e337984 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:13.603022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:13.645412 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:13.646643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:13.646881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:13.647164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:13.647969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:13.649427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:13.695471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.sql.core [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 12:46:13.695471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 12:46:13.695471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [parameters: {'id': 'deb6cc726a9e4d5eb25169d7150559fe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 12:46:13.695471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 12:46:13.710017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67c723-4a7c-4681-8541-ce7f446f6da1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 13 12:46:13.726351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 12:46:13.727276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application yield resource Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise self.value Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application session.commit() Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 12:46:13.728164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.flush() Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application compat.raise_( Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 12:46:13.728996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 12:46:13.729911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 12:46:13.730713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [parameters: {'id': 'deb6cc726a9e4d5eb25169d7150559fe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:13.731476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 12:46:13.732447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:13.733287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 237/477] 10.222.0.72 () {60 vars in 1131 bytes} [Tue Sep 13 12:46:13 2022] POST /identity/v3/roles => generated 144 bytes in 350 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:13.739481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:13.765032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:13.765648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:13.765953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:13.766228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:13.766992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:13.768380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:13.851232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 241/478] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:46:13 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:46:13.870173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f57c7b-26d5-4b77-83b1-cdc5a9d9b4c6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:13.912459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378473b7-747d-40f2-8a2b-37ffae2278e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:13.913264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378473b7-747d-40f2-8a2b-37ffae2278e6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:13.913546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378473b7-747d-40f2-8a2b-37ffae2278e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:13.913840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378473b7-747d-40f2-8a2b-37ffae2278e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:13.926126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378473b7-747d-40f2-8a2b-37ffae2278e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-378473b7-747d-40f2-8a2b-37ffae2278e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:13.931808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:13.932727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:13.933525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 238/479] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:13.940642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:13.976896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:14.087427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e8399-209a-42ed-be7a-95da1b02e1ea None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:14.122949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437a648-6b6f-4fc5-b5b3-418673b23e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:14.123548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437a648-6b6f-4fc5-b5b3-418673b23e74 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:14.123822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437a648-6b6f-4fc5-b5b3-418673b23e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:14.124091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437a648-6b6f-4fc5-b5b3-418673b23e74 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:14.124748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437a648-6b6f-4fc5-b5b3-418673b23e74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:14.125048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437a648-6b6f-4fc5-b5b3-418673b23e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:14.126447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437a648-6b6f-4fc5-b5b3-418673b23e74 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:14.135657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 242/480] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:15.477554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506e303-30da-49ec-99c5-a8b3f1730b55 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:15.478111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506e303-30da-49ec-99c5-a8b3f1730b55 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:15.478111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506e303-30da-49ec-99c5-a8b3f1730b55 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:15.479721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 239/481] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:15.484998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390486-2d08-4678-937d-124edb7f7ec8 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:15.485419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390486-2d08-4678-937d-124edb7f7ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:15.485606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390486-2d08-4678-937d-124edb7f7ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:15.508145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e4390486-2d08-4678-937d-124edb7f7ec8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:15.509332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e4390486-2d08-4678-937d-124edb7f7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:15.603643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 243/482] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:15.736611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c926cd-3009-466d-956f-07ad03a614c2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:15.736923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c926cd-3009-466d-956f-07ad03a614c2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:15.737102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c926cd-3009-466d-956f-07ad03a614c2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:15.738676 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 240/483] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:15.743625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670ed1e-10b3-4baa-bb4b-6ed1e43da9e7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:15.743911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670ed1e-10b3-4baa-bb4b-6ed1e43da9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:15.744187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670ed1e-10b3-4baa-bb4b-6ed1e43da9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:15.765367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1670ed1e-10b3-4baa-bb4b-6ed1e43da9e7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:15.766644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1670ed1e-10b3-4baa-bb4b-6ed1e43da9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:15.774732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:15.854837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 244/484] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:15.909120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c926cd-3009-466d-956f-07ad03a614c2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:15.955408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:16.037725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c926cd-3009-466d-956f-07ad03a614c2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:16.081639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:16.082386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.082666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.082938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.100847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:16.105695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:16.106554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:16.108097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 241/485] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:16.115538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670ed1e-10b3-4baa-bb4b-6ed1e43da9e7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.182079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.286719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670ed1e-10b3-4baa-bb4b-6ed1e43da9e7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:16.348995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:16.353318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:16.355093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.355377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.355648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.356325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:16.356626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:16.358026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:16.362809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d225-8978-4efe-b3ba-4a4efacbad8e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:16.380511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 245/486] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:16.387731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.393315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:16.417242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf0462-456c-4117-80a6-e679e91d52cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:16.417907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf0462-456c-4117-80a6-e679e91d52cd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.418192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf0462-456c-4117-80a6-e679e91d52cd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.418467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf0462-456c-4117-80a6-e679e91d52cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.430471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf0462-456c-4117-80a6-e679e91d52cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cddf0462-456c-4117-80a6-e679e91d52cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:16.435563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:16.436450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:16.437903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 242/487] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:16.438559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f051dfe-445e-49bc-81c5-c070ee99255e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.438559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f051dfe-445e-49bc-81c5-c070ee99255e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.438879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f051dfe-445e-49bc-81c5-c070ee99255e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.439949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 243/488] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:16.442557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f051dfe-445e-49bc-81c5-c070ee99255e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.518496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553d484-296f-4ab6-97ed-ea3cc1f56ef6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:16.553910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff2e2d-97b6-4192-9ad5-9fffc0c7673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:16.554907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff2e2d-97b6-4192-9ad5-9fffc0c7673a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.555397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff2e2d-97b6-4192-9ad5-9fffc0c7673a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.555871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff2e2d-97b6-4192-9ad5-9fffc0c7673a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.561446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f051dfe-445e-49bc-81c5-c070ee99255e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:16.567698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff2e2d-97b6-4192-9ad5-9fffc0c7673a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:16.570001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff2e2d-97b6-4192-9ad5-9fffc0c7673a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-f0ff2e2d-97b6-4192-9ad5-9fffc0c7673a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:16.585209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:16.586167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:16.586993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:16.586993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:16.586993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:16.593247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 246/489] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:16.594714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741976a-24e3-4a61-bd8b-c1b47baaf602 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.595192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741976a-24e3-4a61-bd8b-c1b47baaf602 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.595675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741976a-24e3-4a61-bd8b-c1b47baaf602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.602229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:16.602809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.603092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.603421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.604215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:16.604525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:16.606190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:16.616648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7741976a-24e3-4a61-bd8b-c1b47baaf602 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:16.616968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 244/490] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:16.617804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.618696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7741976a-24e3-4a61-bd8b-c1b47baaf602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:16.643181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:16.687664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:16.705379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 247/491] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:16.706650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741976a-24e3-4a61-bd8b-c1b47baaf602 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.767422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c29caa-8113-463e-9e9e-39e1917c7b2c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:16.806150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8eacc-d865-42e8-85d2-22b6954ad42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:16.806858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8eacc-d865-42e8-85d2-22b6954ad42f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.807364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8eacc-d865-42e8-85d2-22b6954ad42f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.807812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8eacc-d865-42e8-85d2-22b6954ad42f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.808658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8eacc-d865-42e8-85d2-22b6954ad42f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:16.809170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8eacc-d865-42e8-85d2-22b6954ad42f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:16.809765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741976a-24e3-4a61-bd8b-c1b47baaf602 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:16.810911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8eacc-d865-42e8-85d2-22b6954ad42f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:16.824316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 245/492] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:16.825826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3d132-955e-462f-bec6-0296c6206770 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.826556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3d132-955e-462f-bec6-0296c6206770 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.827049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3d132-955e-462f-bec6-0296c6206770 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.832629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 246/493] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:16.834750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a448c-7e30-4653-8f60-bafdfbcc8370 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.835259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a448c-7e30-4653-8f60-bafdfbcc8370 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.835747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a448c-7e30-4653-8f60-bafdfbcc8370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.851623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:16.852311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:16.852527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:16.852953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:16.855763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fe1a448c-7e30-4653-8f60-bafdfbcc8370 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:16.857191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fe1a448c-7e30-4653-8f60-bafdfbcc8370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:16.864332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:16.866401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:16.885245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:16.887021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:16.887850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:16.887850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:16.887850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:16.887850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 248/494] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:16.887850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.930305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:16.938498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 247/495] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:16.939605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a448c-7e30-4653-8f60-bafdfbcc8370 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:16.952016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.037238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f2c03-e7f4-4ada-9810-4cbaa8c0cbf9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.056975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a448c-7e30-4653-8f60-bafdfbcc8370 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.077217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.077927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.078205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.078475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.087242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.088760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.094509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b033b8-981d-4968-b0ff-8ba5294e8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.095119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b033b8-981d-4968-b0ff-8ba5294e8113 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.095400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b033b8-981d-4968-b0ff-8ba5294e8113 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.095672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b033b8-981d-4968-b0ff-8ba5294e8113 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.096313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b033b8-981d-4968-b0ff-8ba5294e8113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:17.096617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b033b8-981d-4968-b0ff-8ba5294e8113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.097987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b033b8-981d-4968-b0ff-8ba5294e8113 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:17.100984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:17.101872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:17.102817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:17.106766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 249/496] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:17.107328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.115867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 248/497] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:17.115867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8340db-1b58-4cb6-af22-d5391f0d22fd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.115867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8340db-1b58-4cb6-af22-d5391f0d22fd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.115867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8340db-1b58-4cb6-af22-d5391f0d22fd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.115867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 249/498] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:17.115867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8340db-1b58-4cb6-af22-d5391f0d22fd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.188283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.206862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.210712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932a47c-31a6-49f2-b09c-69cd400396e8 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.226068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8340db-1b58-4cb6-af22-d5391f0d22fd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.243389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.244037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.244312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.244579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.254797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:17.260362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:17.261245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:17.262065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:17.263541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d1ff6-3d7c-404e-bc09-ef30e9f75335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.264153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d1ff6-3d7c-404e-bc09-ef30e9f75335 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.264488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d1ff6-3d7c-404e-bc09-ef30e9f75335 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.264707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d1ff6-3d7c-404e-bc09-ef30e9f75335 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.265451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d1ff6-3d7c-404e-bc09-ef30e9f75335 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:17.265834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d1ff6-3d7c-404e-bc09-ef30e9f75335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.267206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d1ff6-3d7c-404e-bc09-ef30e9f75335 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.269290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 250/499] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:17.270068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.305134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 250/500] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:17.306407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99da6e-e44e-46bf-8bb0-e3c14191e5f0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.306699 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99da6e-e44e-46bf-8bb0-e3c14191e5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.306979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99da6e-e44e-46bf-8bb0-e3c14191e5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.325260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6b99da6e-e44e-46bf-8bb0-e3c14191e5f0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:17.326729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6b99da6e-e44e-46bf-8bb0-e3c14191e5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:17.367751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e1435-04fc-49c6-85b9-53c3efca2107 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.392948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 251/501] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:17.394066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99da6e-e44e-46bf-8bb0-e3c14191e5f0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.402091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.402725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.403006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.403279 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.411064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.413215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:17.424866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:17.426049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 251/502] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:17.427928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.449465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.462039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.506094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99da6e-e44e-46bf-8bb0-e3c14191e5f0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.538141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acea99c-b41f-4f32-8b78-15e7e216789e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.544179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.545439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.545894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.546307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.547576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:17.548021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.550194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.560968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 252/503] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:17.562513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.575125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.576404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.576686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.577112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.587555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:17.592741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:17.593604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:17.594468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 252/504] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:17.595111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.676816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaf681-8984-4062-a33d-1277d22343f6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.682319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.694481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.702745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2148c4-5016-41e0-abd5-4be44367d0ae None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.710875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.711480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.711761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.712034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.736150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.736746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.739166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.739166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.739166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:17.739166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.739750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.775614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 253/505] 10.222.0.72 () {56 vars in 1137 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:17.776449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.783694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.785542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.872827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a96d19-4a4d-4921-955e-2a3027afb6b5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:17.885295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 253/506] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:17.886097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.904286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.911782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:17.912668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:17.912943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:17.913214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:17.915896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:17.923526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:17.924997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:17.940000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:17.941124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:17.941975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:17.941975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:17.941975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:17.941975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 254/507] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:17.941975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:17.987404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6351a-7266-4a24-bed4-e5e28b8e8569 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.021885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.022496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.022780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.023055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.023725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:18.024031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.025383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.033017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 254/508] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:18.033826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.047782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bcf80-abe7-4c16-a950-b7900eb4b7b0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.080966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.081589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.081896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.082180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.094530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.095971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:18.110427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:18.111355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:18.112202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:18.112202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.112202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:18.112202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 255/509] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:18.112409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.116723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.134716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4b05e-387c-4cf4-bf08-8ad24c542378 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.153977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.169259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.169902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.170191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.170463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.211388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1e032-278c-4f4f-8e97-79dd2e1bdf06 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.238395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.240009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.243254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.243868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.244142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.254666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.256829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:18.270649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:18.272094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:18.272914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:18.272914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.272914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:18.272914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 256/510] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:18.273534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.319448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.340029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 255/511] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:18.340849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.363098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.371983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfae23-137b-45a3-ab3c-eb597ff46e60 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.404611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.405134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.405408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.405675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.406342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:18.406640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.407988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.420005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 257/512] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:18.420803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.442906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579da29-3140-4976-80f3-4094d53858e2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.477059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.477680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.477989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.478259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.488031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.489459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:18.504506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:18.506163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:18.507016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:18.507016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.507016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:18.507016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 256/513] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:18.507016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.522506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b2b42-faa2-4bcb-b1f5-588055ccdd36 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.542215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.555983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.556655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.556939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.557214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.557884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:18.558188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.559593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.576421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 258/514] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:18.576421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.589267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.609722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab26b1-919e-416c-9e5e-090b794fb0a2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.645402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.646077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.646506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.646612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.647259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:18.647563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.648925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.660998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 257/515] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:18.661766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.680294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f50061-6386-4a61-9d46-6b91b4baa206 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.713102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.713723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.714034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.714306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.724605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.726094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:18.740685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:18.741788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:18.742998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:18.742998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:18.742998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:18.742998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 259/516] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:18.742998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.754134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.764370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e30925-df54-4950-9ac3-bc97d5dd2bce None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.797666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.798679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.798962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.799243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.806509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:18.807315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.808767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:18.822166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:18.823070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:18.823953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 258/517] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:18.824510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.845604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b6779-a02d-413b-8c3c-76dce1db99a0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.878505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.879100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.879381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.879650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.880324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:18.881121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.882713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.894284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 260/518] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:18.895175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:18.930211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e52bbf-1d4f-474f-8651-72fc380b7f54 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:18.963687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:18.964300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:18.964591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:18.964855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:18.965498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:18.966973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:18.993695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2c37b-97fe-4096-847d-563ec6bcf093 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.016692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.027366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.027983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.028259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.028527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.034864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.036371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:19.047615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:19.048603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 261/519] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:19.049667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.099756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 259/520] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:46:18 2022] POST /identity/v3/users => generated 247 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:19.100714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.116652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.169968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459fe5e-6705-446b-b14f-fcf83b3cb768 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.207850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.209287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.209698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.210064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.210702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:19.211008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.212418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.225441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 262/521] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:19.226187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.230853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab70de-511d-4005-be81-e42f4e044353 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.268475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.269098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.269381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.269656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.270430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:19.270738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.272679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.277424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.292219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 260/522] 10.222.0.72 () {56 vars in 1137 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:19.293140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.340152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3735b5-8895-4a2a-b799-1d1264ecf716 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.373498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.375938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.376659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.376938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.377209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.385553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.387072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:19.402994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:19.404124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 263/523] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:19.405288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.416759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3889c-fb7e-45d8-91b5-f7969dbdea0c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.453053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.453667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.453979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.454255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.454951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:19.455263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.456690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.481188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 261/524] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:19.482602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.513486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7174c1-e3ca-4cf4-94a0-7acdbea39d2e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.516785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.550004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.550772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.551149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.551524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.558594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.560033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:19.572018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:19.572925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 264/525] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:19.573895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.594999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbce90-227c-49b7-bc6d-8d88bf5ee1dc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.623352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.632081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.637415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.637415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.637415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.693339 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9417c0-cd01-4a8a-b450-67a569d59a25 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.709043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.710763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.731793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.732366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.732645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.732916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.733885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:19.734192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.735598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.759244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 265/526] 10.222.0.72 () {56 vars in 1139 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:19.759244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.784364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.828096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 262/527] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:19.828096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.854115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:19.867789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf48f9d-a452-433b-aeba-2a1fba4cd806 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:19.902962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:19.903563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:19.903840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:19.904116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:19.904839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=3b21d991d02b4f95a8d0fcec0a334495) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:19.905145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:19.906609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:19.913791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 266/528] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/role_assignments?user.id=3b21d991d02b4f95a8d0fcec0a334495&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:19.913791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:19.959572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79979d4d-0d01-404d-b20f-f3adec359738 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.004628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.005969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.006615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.007090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.008015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:20.008552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.010304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.024207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 263/529] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:20.025428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:20.034227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:20.058601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe741a2-68a9-4c0c-985b-680856c4d15f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.104969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.105618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.106039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.106318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.150874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:20.161780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ada4d5-c3f9-473e-94b9-8173357d5ad7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.181188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.182695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.208919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.209824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.210185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.210528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.223092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.225197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:20.244410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:20.245369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:20.246252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:20.246252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:20.246252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:20.250396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 264/530] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:20.251525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:20.273469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:20.294681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 267/531] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:20.295897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:20.360118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0cf8b-c98c-4f84-8218-b23bb7d881d4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.401269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840c7a8-f792-4a97-84a7-8519dcd6bea4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.406246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.407112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.407395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.407668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.408400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=04084ffe1b0548fea75ea144977e4f15) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:20.408708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.410207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.418996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 265/532] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/role_assignments?user.id=04084ffe1b0548fea75ea144977e4f15&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:20.420016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:20.429793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:20.499180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.499815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.500094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.500366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.501025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:20.501352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.502755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.533968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417f869-5558-4c46-9703-9e93dfb0e62f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.535735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 268/533] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:20.551238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:20.576206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.576848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.577132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.577406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.588815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.590315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:20.606661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:20.607623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:20.608521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:20.608521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:20.608521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:20.608521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 266/534] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:20.619696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:20.666132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:20.720008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:20.757430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094a1e2-2a25-4796-90c6-c34ced843f76 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.815183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.816919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.817194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.817465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.823237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1d548-3903-4186-b94b-e741779c9367 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:20.837530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.839229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:20.861012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:20.862454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:20.863371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 269/535] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:20.873865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:20.891850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:20.898308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:20.898308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:20.898308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:20.898308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:20.898308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:20.898308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:20.922393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 267/536] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:20.927751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:21.043791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e57bf-b95a-486f-a6fe-7192e44cb315 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:21.060303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:21.083251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:21.084206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:21.085043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:21.085464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:21.093201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:21.094856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:21.108417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:21.109320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:21.110175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:21.112674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 268/537] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:21.119177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:21.120925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e885674-bea2-4d49-ad61-264065dc429b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:21.138131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:21.197286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:21.197933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:21.198209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:21.198478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:21.199335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:21.199664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:21.201467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:21.221830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 270/538] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:21.230925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:21.302996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896a461-f10c-4313-b741-e6cbf5fce8f3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:21.365802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6f45e-ea8a-4f1f-a9fa-bb6f79c7d985 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:21.403232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5613f1-5711-41da-af4c-b93c986715f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:21.403880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5613f1-5711-41da-af4c-b93c986715f7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:21.404158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5613f1-5711-41da-af4c-b93c986715f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:21.404426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5613f1-5711-41da-af4c-b93c986715f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:21.405155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5613f1-5711-41da-af4c-b93c986715f7 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=6c5b4fdb65514ca6b8033f0f1e4c5eab) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:21.405456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5613f1-5711-41da-af4c-b93c986715f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:21.407121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5613f1-5711-41da-af4c-b93c986715f7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:21.422304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 271/539] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:21 2022] GET /identity/v3/role_assignments?user.id=6c5b4fdb65514ca6b8033f0f1e4c5eab&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:21.423025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:21.423680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:21.423964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:21.424243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:21.425013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:21.425327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:21.426903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:21.451604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41665034-c6d1-47ba-823c-725c3cbdfc53 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:21.452018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41665034-c6d1-47ba-823c-725c3cbdfc53 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:21.452295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41665034-c6d1-47ba-823c-725c3cbdfc53 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:21.453650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 272/540] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:46:21.455916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 269/541] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:21.461888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:21.462270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41568064-beb2-4b81-b673-44da5ce3fd6f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:21.462550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41568064-beb2-4b81-b673-44da5ce3fd6f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:21.462934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41568064-beb2-4b81-b673-44da5ce3fd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:21.498802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-41568064-beb2-4b81-b673-44da5ce3fd6f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:21.502237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-41568064-beb2-4b81-b673-44da5ce3fd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:21.542154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:21.558145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:21.644658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2993a4f-9344-4d1b-af16-62950b17d82a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:21.650396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 273/542] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:21.682903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:21.683507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:21.683787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:21.684054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:21.684777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=4ed50a15319e4991ae90b0e5271f4c91) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:21.685082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:21.686554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:21.692504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 270/543] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:21 2022] GET /identity/v3/role_assignments?user.id=4ed50a15319e4991ae90b0e5271f4c91&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:21.974389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41568064-beb2-4b81-b673-44da5ce3fd6f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:22.103860 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41568064-beb2-4b81-b673-44da5ce3fd6f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:22.153169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:22.153828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:22.154107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:22.154378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:22.167365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:22.172255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:22.173132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:22.173964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 274/544] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:22.179332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:22.308198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:22.314571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86822d2d-4663-4bf3-beb6-974b067fb7e4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:22.357169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53554b22-1f1f-4953-80b6-61c13df52311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:22.357804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53554b22-1f1f-4953-80b6-61c13df52311 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:22.358088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53554b22-1f1f-4953-80b6-61c13df52311 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:22.358363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53554b22-1f1f-4953-80b6-61c13df52311 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:22.359034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53554b22-1f1f-4953-80b6-61c13df52311 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:22.359336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53554b22-1f1f-4953-80b6-61c13df52311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:22.360688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53554b22-1f1f-4953-80b6-61c13df52311 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:22.370212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 271/545] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:22.375685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:22.403857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:22.421642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d60aa-111d-4272-a0de-1e716b16069a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:22.421981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d60aa-111d-4272-a0de-1e716b16069a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:22.422224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d60aa-111d-4272-a0de-1e716b16069a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:22.424192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 272/546] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:22.430068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebec6a4-77f3-4996-94a4-5feba417666c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:22.430339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebec6a4-77f3-4996-94a4-5feba417666c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:22.430610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebec6a4-77f3-4996-94a4-5feba417666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:22.465016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-eebec6a4-77f3-4996-94a4-5feba417666c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:22.466243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-eebec6a4-77f3-4996-94a4-5feba417666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:22.557102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e909d0d-cbc4-495e-a902-5346eea738b9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:22.606779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 273/547] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:46:22.631587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:22.632234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:22.632509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:22.632776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:22.646336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:22.647803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:22.664734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:22.665623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:22.666985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:22.666985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:22.666985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:22.674356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 275/548] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:22.679003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebec6a4-77f3-4996-94a4-5feba417666c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:22.777842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:22.788058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebec6a4-77f3-4996-94a4-5feba417666c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:22.829482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:22.830112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:22.830387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:22.830639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:22.831289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:22.831593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:22.832951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:22.848684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 274/549] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:22.854610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:22.943457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:22.956761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:23.009099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fc1e5-34a1-4516-a2aa-3c972a47d84e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:23.078143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37940af8-f503-4f18-a824-88dce3e19351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:23.078879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37940af8-f503-4f18-a824-88dce3e19351 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.079167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37940af8-f503-4f18-a824-88dce3e19351 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.079447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37940af8-f503-4f18-a824-88dce3e19351 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.090283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37940af8-f503-4f18-a824-88dce3e19351 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:23.092153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37940af8-f503-4f18-a824-88dce3e19351 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:23.099797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907b5fa-a7fe-42da-8c48-fce8c01f34e4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-37940af8-f503-4f18-a824-88dce3e19351 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:23.106784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:23.107882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:23.108979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 276/550] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:23.109622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d02a-fdc1-458f-a519-d7679d74b86b None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.109785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d02a-fdc1-458f-a519-d7679d74b86b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.109991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86d02a-fdc1-458f-a519-d7679d74b86b None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.112034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 277/551] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:23.118671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d02a-fdc1-458f-a519-d7679d74b86b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:23.147983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19156a6e-667e-48e7-a8ea-4c9547ef0242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:23.148886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19156a6e-667e-48e7-a8ea-4c9547ef0242 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.149304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19156a6e-667e-48e7-a8ea-4c9547ef0242 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.150288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19156a6e-667e-48e7-a8ea-4c9547ef0242 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.164402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19156a6e-667e-48e7-a8ea-4c9547ef0242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-19156a6e-667e-48e7-a8ea-4c9547ef0242 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:23.171623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:23.172713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:23.173693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:23.176826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 275/552] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:23.181926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9486-f777-40f4-a1e9-dc681671430b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.181926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9486-f777-40f4-a1e9-dc681671430b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.181926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9486-f777-40f4-a1e9-dc681671430b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.209512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bc1d9486-f777-40f4-a1e9-dc681671430b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:23.211852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bc1d9486-f777-40f4-a1e9-dc681671430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:23.216449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:23.254658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86d02a-fdc1-458f-a519-d7679d74b86b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:23.290182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:23.295347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 276/553] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:23.296211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9486-f777-40f4-a1e9-dc681671430b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:23.296941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:23.297826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.298506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.298968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.302752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:23.302752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:23.302752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:23.332328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 278/554] 10.222.0.72 () {56 vars in 1135 bytes} [Tue Sep 13 12:46:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:23.338032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:23.405787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9486-f777-40f4-a1e9-dc681671430b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:23.447694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:23.448303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.448582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.448855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.450387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:23.450644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:23.452916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:23.469676 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 277/555] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:23.478207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:23.487002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:23.535773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb8a7e9-2a12-4caa-88b8-183a7fa7c9bf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:23.604549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:23.605161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.605437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.607793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.659965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b08f82-567b-421e-8865-ad846790c848 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:23.699818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:23.700461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.700992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.701268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.718372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:23.720691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:23.730297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:23.741124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:23.751379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:23.752351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:23.752351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:23.752351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:23.752528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:23.752621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 278/556] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:23.752690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:23.752764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:23.809093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:23.857948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1f5fe-3b5c-4381-a406-34032b95fc19 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:23.858192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 279/557] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 12:46:23.859225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:23.894947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:23.896229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:23.897991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:23.897991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:23.908343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:23.915792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:23.917195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 279/558] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:23.922942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:23.970290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:23.991448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902525a4-b0a7-4cd7-aea4-123c24e76521 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.023538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.024976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.025428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.025896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.026803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:24.027327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.028873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.033221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab7ee9-e71c-44c2-a2cf-b397844f8656 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.041583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 280/559] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:24.042732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.057943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.083331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.084020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.084232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.084512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.094489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.095921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:24.110566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:24.111719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:24.112791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:24.112791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:24.112791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:24.112791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 280/560] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:24.112791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.169349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fbd2b-f9bf-403d-98ad-ca44cabb67b2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.206674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0041186-60a1-4552-b5aa-13ca7b385be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.207276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0041186-60a1-4552-b5aa-13ca7b385be0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.207552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0041186-60a1-4552-b5aa-13ca7b385be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.207821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0041186-60a1-4552-b5aa-13ca7b385be0 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.209159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0041186-60a1-4552-b5aa-13ca7b385be0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:24.209498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0041186-60a1-4552-b5aa-13ca7b385be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.210911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0041186-60a1-4552-b5aa-13ca7b385be0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.218918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 281/561] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:24.220121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fb0f8-3c73-42c0-8dcb-1c1d4365ec3c None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.220401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fb0f8-3c73-42c0-8dcb-1c1d4365ec3c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.220668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fb0f8-3c73-42c0-8dcb-1c1d4365ec3c None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.225764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 282/562] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:24.225764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fb0f8-3c73-42c0-8dcb-1c1d4365ec3c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.238562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.240215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f65e2-4c6b-4d23-9a86-64bc1e509c36 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.274696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.275232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.275515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.275789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.283363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.284787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:24.296423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:24.298245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 281/563] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:24.300364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.331252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.333039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fb0f8-3c73-42c0-8dcb-1c1d4365ec3c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.366849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcab4a-47c1-40a1-9883-625a20b42729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.367452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcab4a-47c1-40a1-9883-625a20b42729 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.367729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcab4a-47c1-40a1-9883-625a20b42729 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.368001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcab4a-47c1-40a1-9883-625a20b42729 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.368638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcab4a-47c1-40a1-9883-625a20b42729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:24.368944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcab4a-47c1-40a1-9883-625a20b42729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.370425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcab4a-47c1-40a1-9883-625a20b42729 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.382616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 283/564] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:24.383931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc7412-c4e4-4949-8064-64ff1c3a985f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.384219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc7412-c4e4-4949-8064-64ff1c3a985f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.384484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc7412-c4e4-4949-8064-64ff1c3a985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.401583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-90cc7412-c4e4-4949-8064-64ff1c3a985f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:24.402819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-90cc7412-c4e4-4949-8064-64ff1c3a985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:24.405603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce05ab9-306d-4489-84b0-e1d969f04bd6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.439958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.440582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.440863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.441138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.451189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.452629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:24.466970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:24.468004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:24.468874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:24.468874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:24.468874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:24.469041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 284/565] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:24.469435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9933cd-ccfc-450e-868e-cab5f7da6dc9 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.469532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 282/566] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:24.469722 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9933cd-ccfc-450e-868e-cab5f7da6dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.470027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9933cd-ccfc-450e-868e-cab5f7da6dc9 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.470821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.471546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 285/567] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:24.472110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9933cd-ccfc-450e-868e-cab5f7da6dc9 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.475608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.568275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.581527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9933cd-ccfc-450e-868e-cab5f7da6dc9 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.590599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96332b-cf84-436a-8a6d-deab3940c1de None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.619148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.619770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.620050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.620319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.626244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2d8d7-2989-48dd-bf9d-42af9e5be2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.626844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2d8d7-2989-48dd-bf9d-42af9e5be2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.627121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.627242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2d8d7-2989-48dd-bf9d-42af9e5be2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.627399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2d8d7-2989-48dd-bf9d-42af9e5be2b2 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.628126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2d8d7-2989-48dd-bf9d-42af9e5be2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:24.628439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2d8d7-2989-48dd-bf9d-42af9e5be2b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.628615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.629874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2d8d7-2989-48dd-bf9d-42af9e5be2b2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:24.641260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:24.642185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:24.643093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:24.647273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 286/568] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:24.648136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.655837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 283/569] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:24.656972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd1516-4ef7-4606-95d5-0c6bb2b86392 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.657267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd1516-4ef7-4606-95d5-0c6bb2b86392 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.657543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd1516-4ef7-4606-95d5-0c6bb2b86392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.676181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-99cd1516-4ef7-4606-95d5-0c6bb2b86392 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:24.677948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-99cd1516-4ef7-4606-95d5-0c6bb2b86392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:24.715937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.750196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 284/570] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:24.751045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd1516-4ef7-4606-95d5-0c6bb2b86392 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.770403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0f011-af6b-4f9f-aa37-d31cfad9065b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.805283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.806017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.806294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.806679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.807371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:24.807775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.809342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.822597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.823969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 287/571] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:24.824784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.864696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd1516-4ef7-4606-95d5-0c6bb2b86392 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.901460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.902094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.902377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.902650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.903343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:24.903650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.905122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.923027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 285/572] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:24.923027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:24.930094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840add29-e609-467b-8ef0-7ea6f7021ea3 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:24.955782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:24.969415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:24.970076 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:24.970375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:24.970627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:24.982025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:24.983554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:24.999849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:25.000810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:25.001810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:25.001810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:25.001810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:25.006711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 288/573] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:25.007457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.029686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada1782-7e01-45b3-90c9-30357db91da4 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.054629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:25.062975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.063595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.063871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.069786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.074978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:25.079883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:25.080939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 286/574] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:25.082115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.109561 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbf161-0c1e-415e-9a95-48d957dcf8e9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.142558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.143178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.143456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.143727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.151676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.153110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:25.164407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:25.165533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 289/575] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:25.166522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.189586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda03f38-381e-41ec-80d0-3e6fef0647f6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.199994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:25.225664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.226304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.226589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.226867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.227596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=a08e9745841d4716a64700da59eebf04) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:25.227908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.229357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.235112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 287/576] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/role_assignments?user.id=a08e9745841d4716a64700da59eebf04&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:25.235932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.285488 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ad6b1-8df5-42fe-a3f8-92ad706f232e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.288668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:25.325161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.326265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.326545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.326815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.327642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:25.327948 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.329888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.349779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 290/577] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:25.350790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.355636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ca71-df18-4e0c-ad53-302a944b4b59 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.396817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.397459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.397770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.398036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.409785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:25.416077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:25.417064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 288/578] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:25.418107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.460072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:25.467049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c05b17-43df-429b-8a5e-08685f085e6b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.503476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.504087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.504367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.504644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.505901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:25.506210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.507580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.515778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 291/579] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:25.516595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.529144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f3629-d4ce-475d-bd29-3528656dabef None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.540476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:25.563547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.564152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.564436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.564716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.565449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:25.565779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.567229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.592397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 289/580] 10.222.0.72 () {56 vars in 1137 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:25.592397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.628184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabe861-da07-4e61-bbdc-58dd150935f6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.667856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.668487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.668769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.669035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.675842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.677414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:25.689342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:25.690541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:25.691432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 292/581] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:25.691925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.707191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32af361-012f-4f0c-9967-1e54ce10d162 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.714960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:25.744816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.745428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.745830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.746056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.746807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:25.747115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.748464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.763238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 290/582] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:25.764102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.796380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:25.799883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409c197-e70e-4f57-8dc0-47735f9d2a7d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.836215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.836874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.837152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.837486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.848393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.849881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:25.864189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:25.865373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:25.866296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:25.866296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:25.866296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:25.866296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 293/583] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:25.866296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.876875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0799b-dc78-42c2-b008-884cac5b6d8c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:25.914305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:25.914932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:25.915238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:25.915508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:25.925629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:25.927113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:25.941508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:25.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:25.944233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:25.944233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:25.944233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:25.944233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 291/584] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:25.944473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:25.980103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.018132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3fa87-b5ca-4e56-bcd4-dcd52329d53e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.039556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.058165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.058449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108144e-7724-4fd1-bdf0-769e34e6d743 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.058778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.059058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.059331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.060027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:26.060333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.061806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.076289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 294/585] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:26.077121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.103715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.104270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.104547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.104819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.116301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.118096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:26.134012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:26.135030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:26.135954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:26.135954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:26.135954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:26.135954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 292/586] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:26.136611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.190359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261000e7-0e71-4568-a9dd-c9c99df68d47 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.223477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.232533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.233426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.233996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.234226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.238503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:26.238503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.238503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.253191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 295/587] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:26.253354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102a34c-4010-4bad-bb52-419a8dccc3c7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.253473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.288332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.288939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.289496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.289496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.290133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:26.290438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.291848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.293433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.306498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 293/588] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:26.307981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.361142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9253e-34ee-4848-9f67-56901e3918c5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.398009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.398801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.399080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.399355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/3b21d991d02b4f95a8d0fcec0a334495/roles/fd66e13437394d4f98a74978756fb6e0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.415453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d912bd-3819-4f33-97df-e2b30f4f498f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.416920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=3b21d991d02b4f95a8d0fcec0a334495, role_id=fd66e13437394d4f98a74978756fb6e0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.418399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.451010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.451618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.451899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.452173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.452820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:26.453125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.454509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.466140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 294/589] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:26.466944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.507257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 296/590] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:26 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/3b21d991d02b4f95a8d0fcec0a334495/roles/fd66e13437394d4f98a74978756fb6e0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:26.508101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.530629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.574363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.588833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd40f2-ac42-478e-8874-a430501806b6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.629277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.636707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.637236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.637725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.654770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.656731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.661324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1f4df-8aff-4eca-b393-9efdbae9394d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:26.671149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:26.672150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:46:26.673091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:26.680354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 295/591] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:26.680354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.711502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.712557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.713037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.713504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.720906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.722651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:26.736745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:26.737661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:26.738992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 297/592] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:26.739543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.803227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40bb70-89ef-41d1-8e64-454d591af253 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.819151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.851288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.852288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.852667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.853402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.857837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:26.862555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed8c58-b523-4bf9-b00d-633a518a5853 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:26.867847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.869435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:26.885174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:26.886356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 12:46:26.887583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:26.892600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 296/593] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:26.892600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:26.917451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:26.918348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:26.918769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:26.919183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:26.920597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:26.921086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:26.923283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:26.948398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 298/594] 10.222.0.72 () {56 vars in 1137 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:26.949479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.023379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e505bb8-6789-46a4-9950-3b9972294373 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.065378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.066256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.066683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.067089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.068701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:27.069254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.071546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.071670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece559-546a-4a43-9af3-43ae20126252 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.092462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 297/595] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:27.092462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.117862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.118657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.119271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.119391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.120307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:27.120709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.122325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.135104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:27.150056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 299/596] 10.222.0.72 () {56 vars in 1139 bytes} [Tue Sep 13 12:46:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:27.150056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.168153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:27.218545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769fecb-51f6-47db-8ddd-51ad88bd9f1d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.263841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.264766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.265144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.265518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.276232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b33ac-60ef-4fcf-a054-79c342690fdf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.278698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.281198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:27.301059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:27.303144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:27.304039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:27.304039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:27.304039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:27.305913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 298/597] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:27.307112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.330995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.331692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.331970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.332248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/04084ffe1b0548fea75ea144977e4f15/roles/fd66e13437394d4f98a74978756fb6e0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.354094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=04084ffe1b0548fea75ea144977e4f15, role_id=fd66e13437394d4f98a74978756fb6e0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.355600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.409871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:27.425229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749928d-69e4-41eb-a8e6-57304ffb4739 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.425416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 300/598] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:27 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/04084ffe1b0548fea75ea144977e4f15/roles/fd66e13437394d4f98a74978756fb6e0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:27.426299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.488085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8d07d-5f72-45db-b18f-84af3de959d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.489254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d07d-5f72-45db-b18f-84af3de959d4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.489935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d07d-5f72-45db-b18f-84af3de959d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.490507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d07d-5f72-45db-b18f-84af3de959d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.503803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8d07d-5f72-45db-b18f-84af3de959d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.506711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:27.508574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8d07d-5f72-45db-b18f-84af3de959d4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-31e8d07d-5f72-45db-b18f-84af3de959d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:27.528427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:27.529625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:27.530754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:27.530754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:27.530754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:27.532587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 299/599] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:27.534829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff7c6d-a216-4908-afc2-d7323ab025e2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.535282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff7c6d-a216-4908-afc2-d7323ab025e2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.536011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff7c6d-a216-4908-afc2-d7323ab025e2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.538168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 300/600] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:27.539320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff7c6d-a216-4908-afc2-d7323ab025e2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.558043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558cab1-4479-4aff-bf72-8cc3ad9be4d3 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.598844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d9bdf-2b9f-4ddc-a34e-28a9a7f52ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.599447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9bdf-2b9f-4ddc-a34e-28a9a7f52ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.599822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9bdf-2b9f-4ddc-a34e-28a9a7f52ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.600230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d9bdf-2b9f-4ddc-a34e-28a9a7f52ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.601084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9bdf-2b9f-4ddc-a34e-28a9a7f52ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:27.601440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9bdf-2b9f-4ddc-a34e-28a9a7f52ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.602971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d9bdf-2b9f-4ddc-a34e-28a9a7f52ca8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.619503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 301/601] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:27.620773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deabf7d-74e9-4730-bebd-6a20e70eac14 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.621055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deabf7d-74e9-4730-bebd-6a20e70eac14 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.621329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deabf7d-74e9-4730-bebd-6a20e70eac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.641681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2deabf7d-74e9-4730-bebd-6a20e70eac14 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:27.642974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2deabf7d-74e9-4730-bebd-6a20e70eac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:27.695300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff7c6d-a216-4908-afc2-d7323ab025e2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.729731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 302/602] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:27.731011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deabf7d-74e9-4730-bebd-6a20e70eac14 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.744682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.745525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.745900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.746246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.746992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:27.747395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.749103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.760375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:27.771047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 301/603] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:27.776856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.793345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:27.843413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deabf7d-74e9-4730-bebd-6a20e70eac14 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:27.880432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:27.881515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:27.882041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:27.882506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:27.890137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:27.891815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:27.904281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:27.905166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:27.906081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:27.908140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 303/604] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:27.918642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:27.927948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d634f-9e6f-4cfc-b65e-2e131ffc6b6d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.024992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.025645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.025949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.026226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.038790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:28.042090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:28.057682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ebc6c-2c48-439d-b967-c3e5ea305a8d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:28.064816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:28.065716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:28.066660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:28.070050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 302/605] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:28.071122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:28.085202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:28.107439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.108052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.108326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.108611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.109403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:28.109711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:28.111165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:28.124868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 304/606] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:28.125770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:28.156934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:28.216605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15bcfb-6709-469a-a651-751f4f277a04 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.245537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d463bf-5aef-4aee-92fb-ba073830ca3d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.264981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.266026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.266547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.267040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.283451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:28.288716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:28.289447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:28.290436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:28.291315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 303/607] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:28.292131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:28.297329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.297329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.297329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.297329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:28.297329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:28.297755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:28.316791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 305/608] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:28.317821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:28.365056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:28.405374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01cefe-15ce-4381-8287-d3e4af46de1f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.443254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ee8de-e96e-430e-a542-cd50bfbf215a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.447260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.448223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.448713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.449206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/6c5b4fdb65514ca6b8033f0f1e4c5eab/roles/fd66e13437394d4f98a74978756fb6e0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.473153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=6c5b4fdb65514ca6b8033f0f1e4c5eab, role_id=fd66e13437394d4f98a74978756fb6e0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:28.476129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:28.482879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.483620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.483969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.484386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.485123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:28.485495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:28.487014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:28.501221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 306/609] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:28.501221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:28.507917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:28.590479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 304/610] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:28 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/6c5b4fdb65514ca6b8033f0f1e4c5eab/roles/fd66e13437394d4f98a74978756fb6e0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:28.592260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:28.619474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:28.631175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f72fd0-da72-4cf4-b3ce-f379fcfeb1fa None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.705196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c3f06-8c5e-49e8-b6f0-80c6c7e30b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.706180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c3f06-8c5e-49e8-b6f0-80c6c7e30b20 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.706458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c3f06-8c5e-49e8-b6f0-80c6c7e30b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.706733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c3f06-8c5e-49e8-b6f0-80c6c7e30b20 None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/4ed50a15319e4991ae90b0e5271f4c91/roles/fd66e13437394d4f98a74978756fb6e0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.737572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c3f06-8c5e-49e8-b6f0-80c6c7e30b20 None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=4ed50a15319e4991ae90b0e5271f4c91, role_id=fd66e13437394d4f98a74978756fb6e0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:28.739149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c3f06-8c5e-49e8-b6f0-80c6c7e30b20 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:28.792761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3443c45-542b-4058-acfe-1d5d0fb694d7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:28.823996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 307/611] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:28 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/4ed50a15319e4991ae90b0e5271f4c91/roles/fd66e13437394d4f98a74978756fb6e0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:28.834607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:28.835267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.835546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.835920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.851178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:28.851178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:28.867488 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ea96c-ed3a-4371-8fd5-d10994e6ea21 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.867805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ea96c-ed3a-4371-8fd5-d10994e6ea21 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:28.867944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:28.868818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:28.869720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:28.869720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:28.869720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:28.869920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ea96c-ed3a-4371-8fd5-d10994e6ea21 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.869920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 308/613] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:46:28.870078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 305/613] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:28.875200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b083a5-1340-448b-9aa2-e651409c232a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:28.875664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b083a5-1340-448b-9aa2-e651409c232a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:28.876028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b083a5-1340-448b-9aa2-e651409c232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:28.879896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:28.899727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d3b083a5-1340-448b-9aa2-e651409c232a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:28.901879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d3b083a5-1340-448b-9aa2-e651409c232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:28.910198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:28.991656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 309/614] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:29.018269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ab9c3-aa1d-4201-83eb-72c69ad33711 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:29.059470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:29.060110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:29.060398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:29.060673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:29.061341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:29.061646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:29.063085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:29.086055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 306/615] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:29.091257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b083a5-1340-448b-9aa2-e651409c232a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:29.186423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:29.211300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b083a5-1340-448b-9aa2-e651409c232a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:29.258133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:29.258774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:29.259078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:29.259353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:29.273189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:29.274667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:29.289724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:29.291156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:29.292066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 310/616] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:29.298904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:29.311377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:29.458817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a381d-9f72-4c8c-827c-b37ecf0a39e2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:29.473390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:29.495675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d1094-21d6-4766-8ac0-1482d0aba228 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:29.531237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:29.531850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:29.532131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:29.532405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:29.533143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:29.533448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:29.534903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:29.542466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:29.543120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:29.543405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:29.543684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:29.550316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:29.554542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 307/617] 10.222.0.72 () {56 vars in 1135 bytes} [Tue Sep 13 12:46:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:29.560264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:29.571237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:29.576320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:29.577235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:29.578261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 311/618] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:29.585605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:29.752236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6265e5-b0fa-407e-832d-f644407f37cc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:29.776275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca36f4-91c8-4791-ad05-522d3c0dfa5e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:29.824706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:29.825371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:29.825925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:29.826200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:29.839419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:29.847769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:29.850629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:29.855093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:29.862321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:29.862321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:29.862321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:29.862321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:29.862321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:29.862321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:29.876389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:29.877372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:29.878287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:29.878287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:29.878287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:29.878287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 308/619] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:29.878510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 312/620] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:29.884389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:29.885670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:29.993685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:30.025553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ba7bb-694f-4d7a-9652-b9879b06efc5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.039660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c7926-3214-4e90-9680-c64c9af53981 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.064483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3d117-9ad2-4a30-974f-fb63484f6cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.065118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d117-9ad2-4a30-974f-fb63484f6cbe None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.065402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d117-9ad2-4a30-974f-fb63484f6cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.065676 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3d117-9ad2-4a30-974f-fb63484f6cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.078326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3d117-9ad2-4a30-974f-fb63484f6cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.079796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3d117-9ad2-4a30-974f-fb63484f6cbe None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.080617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20a89b-7db0-476a-b765-41045752b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.081928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20a89b-7db0-476a-b765-41045752b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.082419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20a89b-7db0-476a-b765-41045752b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.082885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20a89b-7db0-476a-b765-41045752b4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.083865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20a89b-7db0-476a-b765-41045752b4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:30.084489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20a89b-7db0-476a-b765-41045752b4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.086165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20a89b-7db0-476a-b765-41045752b4f8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-49e3d117-9ad2-4a30-974f-fb63484f6cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:30.094885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:30.095854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:30.096749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:30.096749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:30.096749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:30.096749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 309/621] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:30.097611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335618fb-6066-4898-918c-782fd607597d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.097920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335618fb-6066-4898-918c-782fd607597d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.098190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335618fb-6066-4898-918c-782fd607597d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.102632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 313/623] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:30.107306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 310/623] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:30.107306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335618fb-6066-4898-918c-782fd607597d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.107512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33e3f7-5723-4e96-9995-22374e5bc7aa None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.107512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33e3f7-5723-4e96-9995-22374e5bc7aa None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.108042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33e3f7-5723-4e96-9995-22374e5bc7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.131323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7a33e3f7-5723-4e96-9995-22374e5bc7aa None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:30.132536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7a33e3f7-5723-4e96-9995-22374e5bc7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:30.162816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:30.225823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 314/624] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:30.227129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33e3f7-5723-4e96-9995-22374e5bc7aa None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.251027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335618fb-6066-4898-918c-782fd607597d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.319283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.320270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.321501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.322782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.325327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:30.326131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.327705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.339528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:30.356591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 311/625] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:30.361779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.407162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33e3f7-5723-4e96-9995-22374e5bc7aa None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.463829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.464477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.464757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.465030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.480123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.481766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.502972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609d1c1-8370-4f99-8b3e-804b05c7724d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.504585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:30.505603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:30.506530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:30.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:30.507858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 315/626] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:30.515418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.568549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.569211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.569542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.569796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.577291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.578770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:30.591145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:30.592130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:30.593080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 312/627] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:46:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:30.593679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.648213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7ebef-ad73-41d8-b0e2-dfcbf4753839 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.657681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:30.685545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.686422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.686865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.687300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.688142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:30.688608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.690228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.694481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8e2e-71af-4ce7-8083-6cc253d3e307 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.710400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 316/628] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:30.711548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.730795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.731479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.731767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.732038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.743032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:30.748417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:30.749505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:30.750422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 313/629] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:30.751125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.782099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:30.827299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2133c2-35ab-4e83-8a21-6d356e4bfda9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.861600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e24b45-6794-4cac-be33-ee8db824d268 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:30.864456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.865369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.865824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.866307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.867188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:30.867635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.869317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.888987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 317/630] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:46:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:30.890162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:30.898627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:30.899293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:30.899573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:30.899847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/a08e9745841d4716a64700da59eebf04/roles/fd66e13437394d4f98a74978756fb6e0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:30.915980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=a08e9745841d4716a64700da59eebf04, role_id=fd66e13437394d4f98a74978756fb6e0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:30.917420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:30.922934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.002191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 314/631] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:30 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/a08e9745841d4716a64700da59eebf04/roles/fd66e13437394d4f98a74978756fb6e0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:31.003935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.006823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef5803-d3a3-46c3-b045-a7e82fc0c640 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.058685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.060592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.062044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.063573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.064456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:31.065426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.069115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:31.088172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 318/632] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:31.094914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.167219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.193217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18b09-db61-4778-b0e0-8571a390d135 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.228022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.242714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38721c6a-c9af-4871-ac11-e6a8f86da8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.243683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38721c6a-c9af-4871-ac11-e6a8f86da8ff None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.243965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38721c6a-c9af-4871-ac11-e6a8f86da8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.244246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38721c6a-c9af-4871-ac11-e6a8f86da8ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.244527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1af06-a167-4569-b447-f96c94e9a497 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.289170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3592c-9a1a-4d05-9a36-ad834e1562ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.291985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3592c-9a1a-4d05-9a36-ad834e1562ca None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.291985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3592c-9a1a-4d05-9a36-ad834e1562ca None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.291985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3592c-9a1a-4d05-9a36-ad834e1562ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.302212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3592c-9a1a-4d05-9a36-ad834e1562ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.303890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3592c-9a1a-4d05-9a36-ad834e1562ca None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-80b3592c-9a1a-4d05-9a36-ad834e1562ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:31.319728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:31.320713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:31.321602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:31.321602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:31.321602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:31.321602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 319/633] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:31.322130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59397503-6f4e-40e7-a19a-0e26487ae2dc None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.322379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59397503-6f4e-40e7-a19a-0e26487ae2dc None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.322556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59397503-6f4e-40e7-a19a-0e26487ae2dc None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.324097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 320/634] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:31.329179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59397503-6f4e-40e7-a19a-0e26487ae2dc None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.336950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38721c6a-c9af-4871-ac11-e6a8f86da8ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.338593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38721c6a-c9af-4871-ac11-e6a8f86da8ff None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:31.413377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.438584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59397503-6f4e-40e7-a19a-0e26487ae2dc None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.445263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 315/635] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:31.446385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379162a-6325-4dd4-9c5a-9dbf7010327c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.446873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379162a-6325-4dd4-9c5a-9dbf7010327c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.447555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379162a-6325-4dd4-9c5a-9dbf7010327c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.466579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7379162a-6325-4dd4-9c5a-9dbf7010327c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:31.467771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7379162a-6325-4dd4-9c5a-9dbf7010327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:31.479600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d11d13-0878-4529-9c1d-58d5230e7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.480220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d11d13-0878-4529-9c1d-58d5230e7e53 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.480476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d11d13-0878-4529-9c1d-58d5230e7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.480745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d11d13-0878-4529-9c1d-58d5230e7e53 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.481373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d11d13-0878-4529-9c1d-58d5230e7e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:31.481677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d11d13-0878-4529-9c1d-58d5230e7e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.483103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d11d13-0878-4529-9c1d-58d5230e7e53 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:31.493318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.501166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 321/636] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:31.501166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d6b0-e724-4099-a448-e1b41a3b2d73 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.501166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d6b0-e724-4099-a448-e1b41a3b2d73 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.501166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d6b0-e724-4099-a448-e1b41a3b2d73 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.501166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 322/637] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:31.506172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42d6b0-e724-4099-a448-e1b41a3b2d73 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.542956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 316/638] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:31.547250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379162a-6325-4dd4-9c5a-9dbf7010327c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.614739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42d6b0-e724-4099-a448-e1b41a3b2d73 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.662906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be3319-f7a3-4700-bfa2-f4923fecc124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.664069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be3319-f7a3-4700-bfa2-f4923fecc124 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.664596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be3319-f7a3-4700-bfa2-f4923fecc124 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.665059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be3319-f7a3-4700-bfa2-f4923fecc124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.669045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379162a-6325-4dd4-9c5a-9dbf7010327c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.670591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.678270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be3319-f7a3-4700-bfa2-f4923fecc124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.680677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be3319-f7a3-4700-bfa2-f4923fecc124 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-d1be3319-f7a3-4700-bfa2-f4923fecc124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:31.698802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:31.699788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:31.701021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:31.701021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:31.701021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:31.702812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 323/639] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:31.704201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c62b46-3360-4423-8ab2-08a8a62d9c8d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.704619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c62b46-3360-4423-8ab2-08a8a62d9c8d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.705011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c62b46-3360-4423-8ab2-08a8a62d9c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.710154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.710949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.711383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.711794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.719810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.721948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:31.728384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e3c62b46-3360-4423-8ab2-08a8a62d9c8d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:31.729783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e3c62b46-3360-4423-8ab2-08a8a62d9c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:31.735373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:31.736446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 12:46:31.737613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:31.741761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 317/640] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:31.741761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.796267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.816216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 324/641] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:31.817940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c62b46-3360-4423-8ab2-08a8a62d9c8d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.861373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e1030-2192-48ce-afe8-d77ac218e56c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.905147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.905672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.906022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.906305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.906958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:31.907265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.908626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:31.922925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 318/642] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:31.923774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:31.940406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c62b46-3360-4423-8ab2-08a8a62d9c8d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:31.951459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:31.993382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:31.994017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:31.994291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:31.994559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:31.995385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:31.995640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:31.997060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.022605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 325/643] 10.222.0.72 () {56 vars in 1137 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:32.023449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.058807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b55093-3c8f-4166-be28-1e300f847c1a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.069953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.103189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.103818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.104760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.104760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.114690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:32.120997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:32.122221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:32.123270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:32.126362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 319/644] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:32.127430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.139433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bee080-b6c8-4cc5-8fa2-0ca88b00c454 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.174779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.175336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.175610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.175883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.183438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.184879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:32.196895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:32.197811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:32.198866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:32.203136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 326/645] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:32.203847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.217133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.234974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b454f-26fc-4469-bf3f-a2fa0b9c05bd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.275668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.277156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.277507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.277881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.307331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.310952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea35b2a-812c-48a6-ae49-32bbd9a960cf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.348219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.349411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.349699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.350006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.351030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.352688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.360701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:32.368136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:32.369092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 327/646] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:32.371177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.446823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.471985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 320/647] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:32.471985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.483437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f5fa6-5d75-4435-946b-e76764e96b87 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.520959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.521705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.522018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.522287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.522960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:32.523243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.524617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.532849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 328/648] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:32.533472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.563924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.595483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a0ca1-9119-4672-8dcc-3602fb30bdea None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.640900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.641721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.642031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.642307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.643023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:32.643329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.644890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.651582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec689526-3813-46ec-a753-a9e70915e6b9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.657890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 321/649] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:32.658606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.693004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.693529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.693836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.694110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.694770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:32.695070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.696446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.709095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 329/650] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:32.709095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.711166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.775336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244fa4-ada1-4158-8490-431914edaa43 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.795735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.812229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9cce31-4555-4f6c-ac96-5aeff3dc9bd6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.817490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.817490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.817490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.817490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.824491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.826231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:32.842111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:32.843248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:32.844354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:32.844354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:32.844354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:32.848520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 322/651] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:32.848520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.855200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.855834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.856108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.857383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.867786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.869295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:32.883760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:32.889094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:32.890024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:32.890024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:32.890024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:32.890024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 330/652] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:32.890024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:32.953192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:32.959836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8b1ce-813b-4831-81c0-ce44e7c1b979 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.995061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:32.995663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:32.995848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b3033-887c-4fa8-8333-ed43057da0ab None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:32.995951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:32.996199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:32.996924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=3b21d991d02b4f95a8d0fcec0a334495) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:32.997290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:32.998781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.004723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 323/653] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/role_assignments?user.id=3b21d991d02b4f95a8d0fcec0a334495&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:33.006144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.038886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.039552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.039773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.040042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.043594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:33.053728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.055236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:33.071186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:33.072229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:33.073146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:33.073146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:33.073146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:33.077489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 331/654] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:33.078526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.161704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2668b35-2fbb-4bf6-a3d3-97a5a7fc0868 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.195156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3cb0b-1d06-424f-88b1-586a46fe184f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.213332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.214478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.214763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.215039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.216638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:33.216950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.218387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.233544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 324/655] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:33.237967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.243139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.243817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.244099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.244370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.245034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:33.245354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.246802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.262242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 332/656] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:33.263369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.264217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:33.329087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:33.379174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99a69-3955-4f42-82b6-e4254563892e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.383189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3261b4-9676-4e45-ae7b-9484b960ddd6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.415708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.416312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.416593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.416862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.417500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:33.417826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.419157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c396d-fe10-49ae-be49-d146159f9c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.419275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.419782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c396d-fe10-49ae-be49-d146159f9c69 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.420054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c396d-fe10-49ae-be49-d146159f9c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.420322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c396d-fe10-49ae-be49-d146159f9c69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.427738 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c396d-fe10-49ae-be49-d146159f9c69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.429748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c396d-fe10-49ae-be49-d146159f9c69 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.433882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 325/657] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:33.434939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-8c5c396d-fe10-49ae-be49-d146159f9c69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:33.441200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:33.442151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:33.443581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:33.446030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 333/658] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:33.447649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11aafa-825b-4d6a-b2a3-5c479101bda0 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.448143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11aafa-825b-4d6a-b2a3-5c479101bda0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.448557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11aafa-825b-4d6a-b2a3-5c479101bda0 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.450042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 334/659] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:33.450957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11aafa-825b-4d6a-b2a3-5c479101bda0 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.524306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:33.547146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797da117-1298-4868-84b2-af3916a7ef9f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.556770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11aafa-825b-4d6a-b2a3-5c479101bda0 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.566183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:33.584161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.584793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.585081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.585350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.591402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0d48b-0187-4f37-a807-f18629f9f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.592057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0d48b-0187-4f37-a807-f18629f9f2ab None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.592805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0d48b-0187-4f37-a807-f18629f9f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.593423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.593533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0d48b-0187-4f37-a807-f18629f9f2ab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.594998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.601663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0d48b-0187-4f37-a807-f18629f9f2ab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.603114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0d48b-0187-4f37-a807-f18629f9f2ab None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:33.607357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:33.608771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:33.609981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 326/660] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:33.610591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-16e0d48b-0187-4f37-a807-f18629f9f2ab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:33.620092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:33.621297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:33.622273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 335/661] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:33.622863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfe33-9410-47ac-91b3-0139c82b1dcb None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.623121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfe33-9410-47ac-91b3-0139c82b1dcb None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.623388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfe33-9410-47ac-91b3-0139c82b1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.644207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-3f4dfe33-9410-47ac-91b3-0139c82b1dcb None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:33.645412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3f4dfe33-9410-47ac-91b3-0139c82b1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:33.722521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 336/662] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:33.724409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfe33-9410-47ac-91b3-0139c82b1dcb None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.746403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7fd7c-6bcd-4185-84eb-2ae03f74ad0d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.789199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.789721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.790038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.790547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.791244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:33.791547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.792956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.802378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:33.813843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 327/663] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:33.813843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.839700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:33.845049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfe33-9410-47ac-91b3-0139c82b1dcb None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.889616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.890262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.890544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.890814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.891727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:33.892039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.894061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:33.921895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 337/664] 10.222.0.72 () {56 vars in 1137 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:33.923088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:33.935173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a43cb-7faa-4b36-b630-57c16ba08f0c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:33.972040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:33.973068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:33.974107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:33.974653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:33.975593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:33.976104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:33.977787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.004613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 328/665] 10.222.0.72 () {56 vars in 1139 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:34.006048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.046961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd588c0-cd92-414f-97f3-cdf8ef07f09c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.066002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.086954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.087525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.087856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.088132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.088862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=04084ffe1b0548fea75ea144977e4f15) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:34.089167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:34.090638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.105788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 338/666] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:33 2022] GET /identity/v3/role_assignments?user.id=04084ffe1b0548fea75ea144977e4f15&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:34.105788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.123845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3ca35-7e6b-4d6c-81a3-a8f0488687a1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.129656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.176608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.177977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.178321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.178661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.242780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22482ff2-3ede-4171-9b39-881fed7fef4b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.266539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:34.268485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.293262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.293993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.294337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.294675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.341885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.380745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:34.382376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.399530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 329/667] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:34.400693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.411888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.511130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 339/668] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:34.515890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.537016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b701ba-a2fd-4d28-977d-1c3ba169c04d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.576056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.576684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.576961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.577227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.589179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:34.594819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:34.595764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:34.596604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 330/669] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:34.597300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.618090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.650085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3449c-cc6e-4fd2-a534-a476e92f38db None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.676322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.695829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.697100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.697883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.698477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.713577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:34.716312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.720820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3119043-b549-4aca-b2af-6f10eb7e4c39 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:34.737178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:34.738257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:34.739909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:34.739909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:34.739909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:34.742072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 340/670] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:34.743036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.767882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.769037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.769935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.770411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.784656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:34.787421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:34.809473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:34.810696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:34.811797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:34.811797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:34.811797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:34.818701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 331/671] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:34.819073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.879240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe937b-0fd7-491a-b1e3-4f5b70fb4ad9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.889839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.917847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.918704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.919115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.919635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.920432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:34.920865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:34.922416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.926189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16594a1-5be6-4f8b-a50a-b53d62a603ac None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:34.934351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 341/672] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:34.935564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:34.962913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:34.963724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:34.964144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:34.964508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:34.965243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:34.965640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:34.967189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:34.975344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:34.985674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 332/673] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:34.991242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.049181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d95e-a0d5-4a00-9888-95d88ff0af44 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.087362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.088201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.088747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.089163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.090094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:35.090537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.092041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.092789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561cdb3-e4f6-43d2-a151-cf1fd032834f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.111081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 342/674] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:35.112456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.124056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:35.139871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.139871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.139871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.139871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.145700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.147329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:35.162879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:35.164075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:35.165182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:35.165182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:35.165182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:35.167253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 333/675] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:35.168205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.222549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:35.230263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5f252-796b-4b0c-bbbb-8489b33a5713 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.269835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.270894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.271409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.271885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.274106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccd61-6161-4869-bb8f-4c843df400c5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.280619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.282635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:35.296174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:35.297156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:35.298090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:35.300789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 343/676] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:35.302715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.317399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.318011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.318289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.318615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.325934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.327415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:35.339566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:35.340458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:35.341383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:35.345390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 334/677] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:35.346780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.390325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:35.417607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5e831-4e1f-48e3-b80e-69bbb6ad2aa5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.452475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7411023-1d2f-45b9-a259-cf121898186f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.454381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.455099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.455375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.455646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.456285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:35.456583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.458571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.475882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 344/678] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:35.475882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.487359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06163bc3-6321-479b-98ec-1566f450dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.487970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06163bc3-6321-479b-98ec-1566f450dd06 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.488253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06163bc3-6321-479b-98ec-1566f450dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.488529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06163bc3-6321-479b-98ec-1566f450dd06 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.489262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06163bc3-6321-479b-98ec-1566f450dd06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:35.489567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06163bc3-6321-479b-98ec-1566f450dd06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.491025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06163bc3-6321-479b-98ec-1566f450dd06 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.496703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:35.503930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 335/679] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:35.505790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663f6b-73a6-4000-958f-9255cdb0a8b9 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.506171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663f6b-73a6-4000-958f-9255cdb0a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.506445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c663f6b-73a6-4000-958f-9255cdb0a8b9 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.509777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 336/680] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:35.509777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663f6b-73a6-4000-958f-9255cdb0a8b9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.596885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9f6ee-eebe-4854-8658-ca77167c5195 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.616400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:35.620430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c663f6b-73a6-4000-958f-9255cdb0a8b9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.637419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.638153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.638433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.638707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.639401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:35.639704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.641188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.657809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 345/681] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:35.657809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.663955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e77970-334c-49c2-87e8-3af12d16c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.664583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e77970-334c-49c2-87e8-3af12d16c24d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.664862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e77970-334c-49c2-87e8-3af12d16c24d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.665128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e77970-334c-49c2-87e8-3af12d16c24d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.672594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e77970-334c-49c2-87e8-3af12d16c24d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.674527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e77970-334c-49c2-87e8-3af12d16c24d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d2e77970-334c-49c2-87e8-3af12d16c24d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:35.687009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:35.688030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:35.688959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 337/682] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:35.689542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262445bb-aa50-4c77-9aed-6ef28d567217 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.689542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262445bb-aa50-4c77-9aed-6ef28d567217 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.690159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262445bb-aa50-4c77-9aed-6ef28d567217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.708403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-262445bb-aa50-4c77-9aed-6ef28d567217 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:35.709604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-262445bb-aa50-4c77-9aed-6ef28d567217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:35.743243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:35.768404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336655ed-11c2-4ca8-8287-a504fdeab221 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.792991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 338/683] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:35.794293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262445bb-aa50-4c77-9aed-6ef28d567217 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.812737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:35.813635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:35.814144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:35.814613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:35.815698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=6c5b4fdb65514ca6b8033f0f1e4c5eab) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:35.816160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:35.818301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:35.830239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 346/684] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/role_assignments?user.id=6c5b4fdb65514ca6b8033f0f1e4c5eab&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:35.831461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:35.970981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262445bb-aa50-4c77-9aed-6ef28d567217 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:35.974609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:36.038406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:36.039183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.039556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.039924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.040910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=4ed50a15319e4991ae90b0e5271f4c91) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:36.041309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:36.043177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:36.045480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f744a8-2dfe-4b41-8f3d-a8d826e88f3a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:36.053854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 339/685] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/role_assignments?user.id=4ed50a15319e4991ae90b0e5271f4c91&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:36.090432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:36.091315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.091740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.092160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.093092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:36.093549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:36.095199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:36.105202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:36.109830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:36.119029 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 347/686] 10.222.0.72 () {56 vars in 1135 bytes} [Tue Sep 13 12:46:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:36.125222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:36.258922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf6d61-27ff-462a-88f3-623d2d332c74 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:36.279063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d46146-a294-4384-a515-8b77bfeb487f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:36.299015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:36.299824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.300058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.300333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.313471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:36.318948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:36.320097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:36.321128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 340/687] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:36.326649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:36.333975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b4560-3108-41bf-bf4b-0ac6df4ba439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:36.334312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b4560-3108-41bf-bf4b-0ac6df4ba439 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.334581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b4560-3108-41bf-bf4b-0ac6df4ba439 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.334855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b4560-3108-41bf-bf4b-0ac6df4ba439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.346551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:36.421696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:36.431193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b4560-3108-41bf-bf4b-0ac6df4ba439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:36.433204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b4560-3108-41bf-bf4b-0ac6df4ba439 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:36.450343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61742b18-9fc7-48db-8092-4b91308e0c1c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:36.490644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21280f50-df21-4066-bfe5-561a50dc4604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:36.493794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280f50-df21-4066-bfe5-561a50dc4604 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.493794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280f50-df21-4066-bfe5-561a50dc4604 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.493794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280f50-df21-4066-bfe5-561a50dc4604 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.493794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21280f50-df21-4066-bfe5-561a50dc4604 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:36.493794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21280f50-df21-4066-bfe5-561a50dc4604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:36.494158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21280f50-df21-4066-bfe5-561a50dc4604 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:36.504974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 341/688] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:36.506591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54995090-e1a5-44e7-b825-a9fcdefb932d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.506773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54995090-e1a5-44e7-b825-a9fcdefb932d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.507042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54995090-e1a5-44e7-b825-a9fcdefb932d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.508501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 342/689] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:36.512901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54995090-e1a5-44e7-b825-a9fcdefb932d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:36.573675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 348/690] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:36.579359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef116a-cbf2-40e7-8256-fcad7725f1e3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.579515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef116a-cbf2-40e7-8256-fcad7725f1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.579774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef116a-cbf2-40e7-8256-fcad7725f1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.601567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1bef116a-cbf2-40e7-8256-fcad7725f1e3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:36.602273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1bef116a-cbf2-40e7-8256-fcad7725f1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:36.610347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:36.646204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54995090-e1a5-44e7-b825-a9fcdefb932d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:36.684471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:36.685268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.685642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.686057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.694441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 349/691] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:36.695304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef116a-cbf2-40e7-8256-fcad7725f1e3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:36.706694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:36.708316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:36.734430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:36.741785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:36.742937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:36.744302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:36.744302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:36.744302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:36.746146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 343/692] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:36.750036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:36.867060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3875c-6bf9-4138-8759-2246128f3d05 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:36.891749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:36.901368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:36.902297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:36.902920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:36.903441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:36.904813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:36.905332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:36.906984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:36.920510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 344/693] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:36.926469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:36.937696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef116a-cbf2-40e7-8256-fcad7725f1e3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.004595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.006448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.006733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.007012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.022888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.024377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.038888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce5b19-eb97-4f8c-b843-881ab2ae29c5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:37.045633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:37.046750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:37.047904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:37.047904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:37.047904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:37.053636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 350/694] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:37.053636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.090068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.091051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.091475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.091889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.100487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.102305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:37.115760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:37.116912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:37.118142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:37.121789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 345/695] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:46:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:37.125861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.162157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:37.180844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:37.189238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2555cf-3a7e-492a-b6b6-04aa4cf9424e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.227568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.228196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.228787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.228787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.240754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.242389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.252951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09551f92-8ecb-4c2b-b869-f54a8734842a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:37.256783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:37.257713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:37.258619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:37.258619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:37.258619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:37.258619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 351/696] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:37.260030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.301264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.302450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.302738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.303012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.303658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:37.303962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.305316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.322893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 346/697] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:37.324007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.372727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e477543-37cf-49aa-b953-cdd44d65a627 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.403225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:37.412870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.413795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.417402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.417402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.417402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:37.417402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.417402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.436741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 352/698] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:37.438156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.451705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccfaa8-2711-4c36-9be8-f00daac32aa9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.463068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:37.492275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.492879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.493189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.493460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.494127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:37.494426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.495765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.510358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 347/699] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:37.511453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.547473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2068b7-2db2-46d8-8d1b-9df9209b97e0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.582057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.583854 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.584340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.584812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.591895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.593568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:37.605649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:37.606699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:37.607568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:37.610857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 353/700] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:37.611944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.629716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bedf0-f920-4405-b072-93f150893153 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.669554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.670233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.670474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.670744 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.679251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:37.716958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:37.737350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5604277-e7db-4ae6-b8e4-a294cf9715b1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:37.759259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.761106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.783934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:37.784788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:37.785210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:37.785764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:37.787130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:37.789022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:37.876916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 348/701] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:37.876916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.913512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 354/702] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:46:37 2022] POST /identity/v3/users => generated 254 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:37.914753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:37.991898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:38.062139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8856e-4c83-4a0a-9eaf-50d9ca8e29d0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:38.114622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc594d-073a-4a7d-92e0-af51e9a5b168 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:38.117499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:38.139804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:38.140419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.140762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.141040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.141756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:38.142081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:38.143505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:38.154747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:38.155762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.156039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.156310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.170407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 349/703] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:38.171787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:38.173275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:38.175824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:38.194598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:38.195619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:38.196527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:38.196527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:38.196527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:38.198832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 355/704] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:38.205110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:38.308112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83f71-c7b1-4bcd-8ace-f6e7b65b2ccb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:38.348810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:38.350327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b97b9e-fe5a-4b1f-84e2-779f2c1f96a0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:38.363589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034ba17-5d70-45b1-b06d-d16ed8073a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:38.365257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034ba17-5d70-45b1-b06d-d16ed8073a64 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.366053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034ba17-5d70-45b1-b06d-d16ed8073a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.366614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034ba17-5d70-45b1-b06d-d16ed8073a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.367764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034ba17-5d70-45b1-b06d-d16ed8073a64 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=a08e9745841d4716a64700da59eebf04) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:38.368173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034ba17-5d70-45b1-b06d-d16ed8073a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:38.369622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0034ba17-5d70-45b1-b06d-d16ed8073a64 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:38.377654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 350/705] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:38 2022] GET /identity/v3/role_assignments?user.id=a08e9745841d4716a64700da59eebf04&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:38.378987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad550a-2624-4fd1-b443-8c945d1933fd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.379270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad550a-2624-4fd1-b443-8c945d1933fd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.379541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad550a-2624-4fd1-b443-8c945d1933fd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.382009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 351/706] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:38.389037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:38.389650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.389956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.390232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.390887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:38.391194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:38.392580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:38.395897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d0d07-2b67-4d14-9d6b-148fd2783bec None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.396218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d0d07-2b67-4d14-9d6b-148fd2783bec None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.397128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d0d07-2b67-4d14-9d6b-148fd2783bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.408200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 356/707] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:38.414728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:38.434682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8f3d0d07-2b67-4d14-9d6b-148fd2783bec None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:38.436646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8f3d0d07-2b67-4d14-9d6b-148fd2783bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:38.454195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:38.587060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24d49b-e5e5-4533-9462-f9c9d0f78c77 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:38.588043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 352/708] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:38.627197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05513c6c-26fe-4c93-b819-a21dc45b2fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:38.627955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05513c6c-26fe-4c93-b819-a21dc45b2fda None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.628296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05513c6c-26fe-4c93-b819-a21dc45b2fda None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.628673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05513c6c-26fe-4c93-b819-a21dc45b2fda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.638231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05513c6c-26fe-4c93-b819-a21dc45b2fda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:38.639880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05513c6c-26fe-4c93-b819-a21dc45b2fda None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-05513c6c-26fe-4c93-b819-a21dc45b2fda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:38.656745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:38.657696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:38.658912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 357/709] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:38.663737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d0d07-2b67-4d14-9d6b-148fd2783bec None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:38.786363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d0d07-2b67-4d14-9d6b-148fd2783bec None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:38.804486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822500e-695e-4f19-9b20-8e374698696b None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.805127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822500e-695e-4f19-9b20-8e374698696b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.805127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822500e-695e-4f19-9b20-8e374698696b None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.806794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 358/710] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:38.811797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:38.815160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70250172-0e54-46e1-a401-20b2318846c9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.815442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70250172-0e54-46e1-a401-20b2318846c9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.815711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70250172-0e54-46e1-a401-20b2318846c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.825913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:38.832366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:38.832366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:38.832366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:38.832366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:38.832366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:38.832366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:38.844395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 353/711] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:38.844652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-70250172-0e54-46e1-a401-20b2318846c9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:38.845716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-70250172-0e54-46e1-a401-20b2318846c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:38.850514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:38.944718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 359/712] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:38.945718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70250172-0e54-46e1-a401-20b2318846c9 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:38.957683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:38.987846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f4ef9-963d-4811-8a8f-293426e32875 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:39.063807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:39.064721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:39.069039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:39.069542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:39.070582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=3b21d991d02b4f95a8d0fcec0a334495) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:39.071119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:39.072793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:39.092368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70250172-0e54-46e1-a401-20b2318846c9 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:39.104692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 354/713] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:38 2022] GET /identity/v3/role_assignments?user.id=3b21d991d02b4f95a8d0fcec0a334495&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:39.152432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:39.153766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:39.154509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:39.155042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:39.155937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:39.157624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:39.178593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:39.180530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:39.181569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:39.184658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 360/714] 10.222.0.72 () {56 vars in 1129 bytes} [Tue Sep 13 12:46:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:39.190426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:39.256122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:39.304483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:39.328764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb37429-3c1b-42c5-bd93-97fbff09b028 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:39.372925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:39.373528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:39.373854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:39.374130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:39.374934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:39.375242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:39.376753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:39.382823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 355/715] 10.222.0.72 () {56 vars in 1124 bytes} [Tue Sep 13 12:46:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:39.388248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:39.466139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:39.497907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecbc65-7115-4028-a114-8352ffa3575a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:39.509557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:39.534427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:39.535047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:39.535328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:39.535603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:39.536277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:39.536579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:39.537398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575064ba-a55d-411c-9e36-679a5b7ad1df None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:39.538248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:39.545505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 356/716] 10.222.0.72 () {56 vars in 1124 bytes} [Tue Sep 13 12:46:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:39.625305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:39.625913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:39.626195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:39.626470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:39.627063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:39.628443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:39.641801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:39.642804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:39.643631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 361/717] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:39 2022] GET /identity/v3/services/network => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:39.649320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:39.762189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7d7e-1453-42c0-b9c3-59142b937837 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:39.815794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3ed02-4fc2-451d-8dc8-a842f44e5ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:39.816566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ed02-4fc2-451d-8dc8-a842f44e5ade None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:39.816795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ed02-4fc2-451d-8dc8-a842f44e5ade None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:39.817071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ed02-4fc2-451d-8dc8-a842f44e5ade None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:39.818420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3ed02-4fc2-451d-8dc8-a842f44e5ade None admin] RBAC: Adding query filter params (name=network) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:39.818723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3ed02-4fc2-451d-8dc8-a842f44e5ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:39.820098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3ed02-4fc2-451d-8dc8-a842f44e5ade None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:39.825215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 357/718] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:46:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:39.830775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:40.027652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be92c6-a874-4add-b544-81ad706b59f7 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:40.076112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b864346-21a1-424d-b634-6e5e8eb0fb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:40.076728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b864346-21a1-424d-b634-6e5e8eb0fb90 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:40.076998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b864346-21a1-424d-b634-6e5e8eb0fb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:40.077269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b864346-21a1-424d-b634-6e5e8eb0fb90 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:40.077922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b864346-21a1-424d-b634-6e5e8eb0fb90 None admin] RBAC: Adding query filter params (type=network) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:40.078220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b864346-21a1-424d-b634-6e5e8eb0fb90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:40.079575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b864346-21a1-424d-b634-6e5e8eb0fb90 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:40.081011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:40.087533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 362/719] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:46:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:40.863363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1cc6b-49ec-47f2-89a5-a81665527294 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:40.865819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1cc6b-49ec-47f2-89a5-a81665527294 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:40.865819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1cc6b-49ec-47f2-89a5-a81665527294 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:40.866670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 358/720] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:40.872133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c29dd4-7331-4ba4-9162-0ecaae0c1a1b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:40.872970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c29dd4-7331-4ba4-9162-0ecaae0c1a1b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:40.873450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c29dd4-7331-4ba4-9162-0ecaae0c1a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:40.893575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-79c29dd4-7331-4ba4-9162-0ecaae0c1a1b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:40.895325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-79c29dd4-7331-4ba4-9162-0ecaae0c1a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:41.000400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 363/721] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:41.509427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cfebe-f4d7-4860-84e3-eaea1e3602f5 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:41.509427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cfebe-f4d7-4860-84e3-eaea1e3602f5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:41.509427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cfebe-f4d7-4860-84e3-eaea1e3602f5 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:41.509427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 359/722] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:41.514901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573b553-16e9-446b-82d3-4de2bf25efb9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:41.515186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573b553-16e9-446b-82d3-4de2bf25efb9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:41.515464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573b553-16e9-446b-82d3-4de2bf25efb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:41.540464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-c573b553-16e9-446b-82d3-4de2bf25efb9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:41.541662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c573b553-16e9-446b-82d3-4de2bf25efb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:41.550313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cfebe-f4d7-4860-84e3-eaea1e3602f5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:41.597456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:41.675079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 364/723] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:41.723067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cfebe-f4d7-4860-84e3-eaea1e3602f5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:41.775403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:41.775961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:41.776341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:41.776619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:41.793594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:41.798915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:41.799864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:41.800735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:41.801394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 360/724] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:41.806846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573b553-16e9-446b-82d3-4de2bf25efb9 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:41.871739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:41.932388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573b553-16e9-446b-82d3-4de2bf25efb9 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:41.983183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:41.984189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:41.984680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:41.985171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:41.986082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:41.986588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:41.988162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:41.998101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 365/725] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:42.003498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.105815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.142046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44322806-6f67-4615-8900-20e56aec9942 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.143946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:42.184342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58233d-ff7b-48ac-8c9d-0b468c0a9e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:42.184975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58233d-ff7b-48ac-8c9d-0b468c0a9e66 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.185256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58233d-ff7b-48ac-8c9d-0b468c0a9e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.185527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58233d-ff7b-48ac-8c9d-0b468c0a9e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.197437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58233d-ff7b-48ac-8c9d-0b468c0a9e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:42.198920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58233d-ff7b-48ac-8c9d-0b468c0a9e66 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7c58233d-ff7b-48ac-8c9d-0b468c0a9e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:42.214683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:42.215694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:42.216662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:42.216662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:42.216662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:42.216662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 361/726] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:42.217338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569f744-6e8b-42e8-85d0-f9b163f35e73 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.217598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569f744-6e8b-42e8-85d0-f9b163f35e73 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.217909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569f744-6e8b-42e8-85d0-f9b163f35e73 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.223266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 362/727] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:42.223266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569f744-6e8b-42e8-85d0-f9b163f35e73 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.277303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eac8e-bbd9-4ac3-805f-00a708211cb1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.299882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:42.315535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec5a56-04b0-4ab8-8146-23617cd609ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:42.316308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec5a56-04b0-4ab8-8146-23617cd609ef None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.316489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec5a56-04b0-4ab8-8146-23617cd609ef None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.316747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec5a56-04b0-4ab8-8146-23617cd609ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.330328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec5a56-04b0-4ab8-8146-23617cd609ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-4cec5a56-04b0-4ab8-8146-23617cd609ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:42.336572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:42.337765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:42.338873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:42.346114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 366/728] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:42.346114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc530c-4344-4489-b43a-622ed175141c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.346114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc530c-4344-4489-b43a-622ed175141c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.346114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc530c-4344-4489-b43a-622ed175141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.357107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569f744-6e8b-42e8-85d0-f9b163f35e73 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.364229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-3dbc530c-4344-4489-b43a-622ed175141c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:42.365808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3dbc530c-4344-4489-b43a-622ed175141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:42.400330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:42.401223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.401653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.402138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.402973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:42.403459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:42.405317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:42.424344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 363/729] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:42.426055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.443046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 367/730] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:42.445941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc530c-4344-4489-b43a-622ed175141c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.447934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:42.545673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75db87-f32e-4dbe-b044-1bddf90bc8ad None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.573027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc530c-4344-4489-b43a-622ed175141c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.584373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8eb7e-6693-4e71-b606-6439e04c37e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:42.585238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8eb7e-6693-4e71-b606-6439e04c37e8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.585799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8eb7e-6693-4e71-b606-6439e04c37e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.586264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8eb7e-6693-4e71-b606-6439e04c37e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.587216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:42.588405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8eb7e-6693-4e71-b606-6439e04c37e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:42.588917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8eb7e-6693-4e71-b606-6439e04c37e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:42.590525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8eb7e-6693-4e71-b606-6439e04c37e8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:42.599553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 364/731] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:42.601123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b0719-242e-471b-8b96-9b6939c877fa None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.601614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b0719-242e-471b-8b96-9b6939c877fa None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.602117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b0719-242e-471b-8b96-9b6939c877fa None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.603788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 365/732] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:42.608934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21cde7-71fe-4c65-afc9-eba826b5ca97 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.609162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21cde7-71fe-4c65-afc9-eba826b5ca97 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.609433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21cde7-71fe-4c65-afc9-eba826b5ca97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.621895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:42.623004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.623376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.623740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.632691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:42.634515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:42.635220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3f21cde7-71fe-4c65-afc9-eba826b5ca97 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:42.636946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3f21cde7-71fe-4c65-afc9-eba826b5ca97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:42.652164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:42.653159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:46:42.654161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:42.658457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 368/733] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:42.659353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.734013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 366/734] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 12:46:42.734712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21cde7-71fe-4c65-afc9-eba826b5ca97 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.737344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:42.790597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1214f-b82a-4fe1-be47-d571072b2e7d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.831312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:42.831976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.832321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.832593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.846398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:42.848089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:42.857564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21cde7-71fe-4c65-afc9-eba826b5ca97 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:42.863423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:42.865210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:42.866217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:42.866217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:42.866217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:42.866217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 369/735] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:42.866217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.892517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:42.897446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86708980-d8e0-41de-afb4-c93864dd70c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:42.898098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86708980-d8e0-41de-afb4-c93864dd70c8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.898441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86708980-d8e0-41de-afb4-c93864dd70c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.898725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86708980-d8e0-41de-afb4-c93864dd70c8 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.899460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86708980-d8e0-41de-afb4-c93864dd70c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:42.899775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86708980-d8e0-41de-afb4-c93864dd70c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:42.901187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86708980-d8e0-41de-afb4-c93864dd70c8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:42.924044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 367/736] 10.222.0.72 () {56 vars in 1149 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:42.924044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462330-7aa5-461a-b66a-aef5baee8e24 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:42.924044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462330-7aa5-461a-b66a-aef5baee8e24 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:42.924044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7462330-7aa5-461a-b66a-aef5baee8e24 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:42.924044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 368/737] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:42.924044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7462330-7aa5-461a-b66a-aef5baee8e24 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:42.978284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94837d51-cd2f-4a4d-805d-b46705840efd None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:42.991557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.012586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.013212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.013492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.013794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.024623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:43.029912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:43.030917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 370/738] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:43.032245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.034967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7462330-7aa5-461a-b66a-aef5baee8e24 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.070623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90fe8d-026f-4f97-aa19-64bc6f9eb035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.071228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90fe8d-026f-4f97-aa19-64bc6f9eb035 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.071508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90fe8d-026f-4f97-aa19-64bc6f9eb035 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.071780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90fe8d-026f-4f97-aa19-64bc6f9eb035 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.072420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90fe8d-026f-4f97-aa19-64bc6f9eb035 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:43.072722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90fe8d-026f-4f97-aa19-64bc6f9eb035 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.074110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90fe8d-026f-4f97-aa19-64bc6f9eb035 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.085572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 369/739] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:43.086765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634456-4f26-412d-bde1-48c48da56983 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.087058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634456-4f26-412d-bde1-48c48da56983 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.087332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634456-4f26-412d-bde1-48c48da56983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.105577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3e634456-4f26-412d-bde1-48c48da56983 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:43.106808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3e634456-4f26-412d-bde1-48c48da56983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:43.118564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.136251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c9382-83a8-4473-9a27-e4789675f0ce None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.169232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.169861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.170142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.170420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.171812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 370/740] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 12:46:43.172625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634456-4f26-412d-bde1-48c48da56983 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.223422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.260381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.261876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.279517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634456-4f26-412d-bde1-48c48da56983 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.315290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.315903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.316449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.316449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.317597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.318988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.346166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.364728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 371/741] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 12:46:43.365562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.410361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 371/742] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:46:43 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:43.411509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.485166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe414-fdac-4e2f-ad78-e7c4e5392df2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.531296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.531924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.532202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.532477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.533150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:43.533453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.535045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.536578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407594a-c480-4048-b9f2-4eca44af4756 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.548778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.549338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 372/743] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:43.550344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.577030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.577689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.577994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.578282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.588125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.589577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.598116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:43.602964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:43.603942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 372/744] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:43.606324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.666234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef4461-b979-4467-9ed6-46a94e04bad2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.703026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.703660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.703938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.704202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.717610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.719105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.724430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8317ba8-7ba4-4df3-bae4-cbcd3e55c82d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:43.733798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:43.734739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:43.735615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:43.735615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:43.735615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:43.735615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 373/745] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:43.735846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.763095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.763707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.763984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.764252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.764896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.766278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.831965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.841027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51aef8-0a92-4e2e-97ce-c9cb34541530 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:43.849906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 373/746] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:46:43 2022] POST /identity/v3/services => generated 240 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:43.849906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.890283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:43.891645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:43.892108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:43.892702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:43.898787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:43.910555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:43.912074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:43.935074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:43.936056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:43.937158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:43.937158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:43.937158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:43.939105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 374/747] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:43.940198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:43.991434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d472-4bdc-4b4c-8c93-86e0f6a08598 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.032111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.032854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.033199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.033713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.034555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:44.034928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.037331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.063294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 374/748] 10.222.0.72 () {56 vars in 1135 bytes} [Tue Sep 13 12:46:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:44.064465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.077066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db92ae-b708-4c1e-a86d-a2d5b32862b9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.117858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.118743 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.119182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.119598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.120443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:44.120901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.122305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.138951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 375/749] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:44.139348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.140314 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:44.185453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:44.196008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286fe24-4667-4309-aa18-f08e78379104 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.240489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.242241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.242789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.243325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.244319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:44.244863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.246467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.246701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa7567-47a7-4357-9e09-67127b6f363c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.265893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 375/750] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:44.265893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.290214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.291728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.292224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.292974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.385492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:44.387419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c668a-5c45-4cff-9de6-38edc299c244 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.388816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.391572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.430802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.431550 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.431896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.432241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.439828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.441318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:44.454848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:44.455783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:44.456679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:44.460896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 376/751] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:44.462156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.475222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:44.513200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 376/752] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:46:44.514433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.582059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09010f-bb07-4c3a-99b6-04073918c770 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.613939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af452728-bec6-446a-9b6f-7681cdba23ad None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.618039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:44.620670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.621305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.621588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.621985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.631110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.632905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:44.646395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:44.656659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 377/753] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:44.657660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.661644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.662589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.663061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.663550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.664438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:44.664944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.666646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.680451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 377/754] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:44.681603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.737081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:44.771689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533ad67-c3f4-42c5-b37d-6a7e15634e7a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.811019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.812177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.812856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.813306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.820297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066174-948e-4d41-869f-ab7164b31cb0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:44.828642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.830764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:44.852192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:44.853405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:44.854486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:44.854486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:44.854486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:44.855122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 378/755] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:44.855991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.866177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:44.866894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:44.867171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:44.867443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:44.868197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:44.868496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:44.870120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:44.894366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 378/756] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:44.894366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:44.912022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:44.998377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175ed45-a764-4dbe-a03f-15aea1a4af40 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.001872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01bdc-6e88-4a32-9b48-d121b3988e2e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.029003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:45.035552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.036398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.036880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.037346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.038237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:45.038734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:45.040285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:45.044989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.050231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.050231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.050231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.050231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=e658441f5ed34cfbb926364644fbaed6) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:45.050231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:45.050231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:45.056573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 379/757] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:45.062400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 379/758] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:44 2022] GET /identity/v3/role_assignments?user.id=e658441f5ed34cfbb926364644fbaed6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:45.062782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:45.062888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:45.230465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:45.273250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f14d8-2d97-4621-b4e1-d1e3529e6625 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.281531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ad48b-e936-496a-aa5a-961a900e93d2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.334372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.335040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.335288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.335558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.338776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.339404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.339681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.339989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.352391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:45.356087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:45.359633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:45.361061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:45.372786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:45.373686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:45.374604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:45.377885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 380/759] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:45.380833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:45.399952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:45.400943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:45.401877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:45.401877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:45.401877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:45.410174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 380/760] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:45.415724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:45.430078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:45.501249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e045b09-c941-4ff5-a7d5-b5fd125269aa None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.539411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.539941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.540221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.540495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.541188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:45.541545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:45.543139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:45.561218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 381/761] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:45.562460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:45.681262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e58d5-57b3-4b99-9ec8-003a3c78788e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.684099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:45.700040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f57c7-a64c-4171-a5b6-07a351ace04e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.721179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b01ccb-73b9-43a1-9aa4-006cae113a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.721815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b01ccb-73b9-43a1-9aa4-006cae113a10 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.722098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b01ccb-73b9-43a1-9aa4-006cae113a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.722369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b01ccb-73b9-43a1-9aa4-006cae113a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.723099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b01ccb-73b9-43a1-9aa4-006cae113a10 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=a08e9745841d4716a64700da59eebf04) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:45.724015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b01ccb-73b9-43a1-9aa4-006cae113a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:45.725428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b01ccb-73b9-43a1-9aa4-006cae113a10 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:45.737896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 382/762] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:45 2022] GET /identity/v3/role_assignments?user.id=a08e9745841d4716a64700da59eebf04&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:45.739548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8aaf-513a-404a-8442-aeab944e216c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.739831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8aaf-513a-404a-8442-aeab944e216c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.740101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8aaf-513a-404a-8442-aeab944e216c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.742851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 383/763] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:45.747322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fb8e6-72b5-4321-bb0c-d94b3b800e3e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.747608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fb8e6-72b5-4321-bb0c-d94b3b800e3e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.747864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fb8e6-72b5-4321-bb0c-d94b3b800e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.754342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:45.775355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.775988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.776268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.776544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:45.777327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:45.777591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:45.778565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fd0fb8e6-72b5-4321-bb0c-d94b3b800e3e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:45.779005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:45.780185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fd0fb8e6-72b5-4321-bb0c-d94b3b800e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:45.810174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 381/764] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:45.811772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:45.890152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 384/765] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:45.957823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4a56c-2ea6-4241-86e1-56b8d876c67d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:45.995181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:45.996221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:45.996695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:45.997157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.005056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:46.006833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:46.018890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:46.019785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:46.021475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:46.024097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 382/766] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:46.029364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fb8e6-72b5-4321-bb0c-d94b3b800e3e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:46.159329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:46.186713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:46.193660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fb8e6-72b5-4321-bb0c-d94b3b800e3e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:46.264145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:46.265395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:46.267290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:46.268041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.269807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:46.270972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:46.272277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:46.274023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:46.292903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 385/767] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:46.298328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:46.389711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1b93d-ead0-4fa1-96e6-f895fa7eb9bf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:46.417166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077eaf8-eb4a-4d82-b26c-278c0a2397c5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:46.424152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebc5b4-2df3-4877-9ddd-4de34419e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:46.424784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebc5b4-2df3-4877-9ddd-4de34419e425 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:46.425068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebc5b4-2df3-4877-9ddd-4de34419e425 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:46.425339 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebc5b4-2df3-4877-9ddd-4de34419e425 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.425954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebc5b4-2df3-4877-9ddd-4de34419e425 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:46.427869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebc5b4-2df3-4877-9ddd-4de34419e425 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-70ebc5b4-2df3-4877-9ddd-4de34419e425 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:46.436787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:46.437805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:46.438661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:46.441850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 383/768] 10.222.0.72 () {56 vars in 1129 bytes} [Tue Sep 13 12:46:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:46.443727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2613b-7087-4ed1-8b16-f7062c325860 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:46.444008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2613b-7087-4ed1-8b16-f7062c325860 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:46.444281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2613b-7087-4ed1-8b16-f7062c325860 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.447130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 384/769] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:46.447998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2613b-7087-4ed1-8b16-f7062c325860 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:46.462379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58769e47-ad6d-490c-b43b-6e03e15c214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:46.463366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58769e47-ad6d-490c-b43b-6e03e15c214c None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:46.463878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58769e47-ad6d-490c-b43b-6e03e15c214c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:46.464663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58769e47-ad6d-490c-b43b-6e03e15c214c None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/3b21d991d02b4f95a8d0fcec0a334495/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.467713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58769e47-ad6d-490c-b43b-6e03e15c214c None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=3b21d991d02b4f95a8d0fcec0a334495, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:46.469557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58769e47-ad6d-490c-b43b-6e03e15c214c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:46.549886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 386/770] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:46 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/3b21d991d02b4f95a8d0fcec0a334495/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 12:46:46.552759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523baf2-7b6a-4d13-a6f6-1c073e2d50c9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:46.553381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523baf2-7b6a-4d13-a6f6-1c073e2d50c9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:46.553903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523baf2-7b6a-4d13-a6f6-1c073e2d50c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.555877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2613b-7087-4ed1-8b16-f7062c325860 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:46.578558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a523baf2-7b6a-4d13-a6f6-1c073e2d50c9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:46.581070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a523baf2-7b6a-4d13-a6f6-1c073e2d50c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:46.591742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:46.602874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:46.643310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:46.643310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:46.643310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:46.643310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.643742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:46.646163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:46.657919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:46.668359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 385/771] 10.222.0.72 () {56 vars in 1124 bytes} [Tue Sep 13 12:46:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:46.674072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:46.684540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 387/772] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:46.822066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82949c9a-5de6-4850-99b3-007e81121eb7 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:46.862612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:46.863268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:46.864709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:46.864709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:46.864881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:46.865185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:46.866625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:46.872242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 386/773] 10.222.0.72 () {56 vars in 1124 bytes} [Tue Sep 13 12:46:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:46.877904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523baf2-7b6a-4d13-a6f6-1c073e2d50c9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:46.977701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:47.011511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523baf2-7b6a-4d13-a6f6-1c073e2d50c9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:47.024929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:47.079594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:47.080271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.080556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.080828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.081533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] RBAC: Adding query filter params (interface=public, service_id=d300a7376f3a4ddbada271d2fd8fd630, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:47.081862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:47.083858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:47.089481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc3847-a80f-4ec3-b688-8968ab0b7808 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:47.102629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 388/774] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:46:46 2022] GET /identity/v3/endpoints?service_id=d300a7376f3a4ddbada271d2fd8fd630&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:47.107823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:47.130920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:47.134359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56df561-de98-4171-998a-235104e68635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:47.134998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56df561-de98-4171-998a-235104e68635 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.135276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56df561-de98-4171-998a-235104e68635 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.135541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56df561-de98-4171-998a-235104e68635 None admin] PATH_INFO: `/v3/services/network` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.137362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56df561-de98-4171-998a-235104e68635 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:47.138770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56df561-de98-4171-998a-235104e68635 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c56df561-de98-4171-998a-235104e68635 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:47.156834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:47.157913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:47.158744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:47.159482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 387/775] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:46 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:47.165904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56df561-de98-4171-998a-235104e68635 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:47.311943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ef453-9d6a-4220-b5b4-30badb1db71f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:47.338584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56df561-de98-4171-998a-235104e68635 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:47.346261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:47.382013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:47.382648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.382930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.383208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.383844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] RBAC: Adding query filter params (name=network) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:47.384149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:47.385522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:47.394003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:47.394615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.394895 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.395331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.396475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:47.398540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:47.401362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 388/776] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:46:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:47.408758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:47.421762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 389/777] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:46:47 2022] GET /identity/v3/services => generated 772 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:47.526667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f2ed-e533-4444-8acf-113fea8e1f10 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:47.565670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873acca-96a6-410a-931d-a72733261c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:47.566309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873acca-96a6-410a-931d-a72733261c37 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.566591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873acca-96a6-410a-931d-a72733261c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.566887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873acca-96a6-410a-931d-a72733261c37 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.567750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873acca-96a6-410a-931d-a72733261c37 None admin] RBAC: Adding query filter params (type=network) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:47.568054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873acca-96a6-410a-931d-a72733261c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:47.569446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873acca-96a6-410a-931d-a72733261c37 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:47.576036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 389/778] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:46:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:47.582052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:47.795581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb2cf-9452-471d-98f4-94f1be4dc87c None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.795989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb2cf-9452-471d-98f4-94f1be4dc87c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.796173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cb2cf-9452-471d-98f4-94f1be4dc87c None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.799100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 390/779] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:46:47.804021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:47.804387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6584-bbee-4aa8-b180-e1dbf6a60534 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.804671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6584-bbee-4aa8-b180-e1dbf6a60534 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.804945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6584-bbee-4aa8-b180-e1dbf6a60534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.829170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5e4d6584-bbee-4aa8-b180-e1dbf6a60534 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:47.830321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5e4d6584-bbee-4aa8-b180-e1dbf6a60534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:47.836547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9977ff-e479-49f9-b3ec-99fdae3a4de7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:47.876247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:47.877166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:47.877508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:47.878007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:47.878495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:47.879416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] RBAC: Adding query filter params (interface=public, service_id=98a7aa63ddab4270876475bf8e1bd797, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:47.880514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:47.882308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:47.888169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 390/780] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:46:47 2022] GET /identity/v3/endpoints?service_id=98a7aa63ddab4270876475bf8e1bd797&interface=public®ion_id=RegionOne => generated 194 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:47.894732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:47.937853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 391/781] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:48.038423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a81610-3ada-48b2-93f5-3599ca36cf94 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:48.088873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96607e7a-5be3-4700-89e2-b533b485e44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:48.089391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607e7a-5be3-4700-89e2-b533b485e44b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:48.089673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607e7a-5be3-4700-89e2-b533b485e44b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:48.089972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96607e7a-5be3-4700-89e2-b533b485e44b None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:48.090577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96607e7a-5be3-4700-89e2-b533b485e44b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:48.091928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96607e7a-5be3-4700-89e2-b533b485e44b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:48.097505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 391/782] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:46:47 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:48.281695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d6584-bbee-4aa8-b180-e1dbf6a60534 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:48.343569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30093ed-e663-438a-a06b-a67d05b5f602 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:48.344500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30093ed-e663-438a-a06b-a67d05b5f602 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:48.344973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30093ed-e663-438a-a06b-a67d05b5f602 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:48.347075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 392/783] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:46:48.353966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444654-3c28-4f40-85cf-d56e6b871bc6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:48.354554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444654-3c28-4f40-85cf-d56e6b871bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:48.355043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da444654-3c28-4f40-85cf-d56e6b871bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:48.370550 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:48.377125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-da444654-3c28-4f40-85cf-d56e6b871bc6 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:48.378582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-da444654-3c28-4f40-85cf-d56e6b871bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:48.418271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d6584-bbee-4aa8-b180-e1dbf6a60534 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:48.468584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 393/784] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:46:48.476302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:48.477127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:48.477617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:48.478080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:48.488847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:48.494704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:48.495824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:48.496872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:48.500032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 392/785] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:48.506055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da444654-3c28-4f40-85cf-d56e6b871bc6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:48.614990 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da444654-3c28-4f40-85cf-d56e6b871bc6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:48.653940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:48.654554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:48.654835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:48.655110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:48.655780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:48.656082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:48.657438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:48.658864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:48.667449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 394/786] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:48.672951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:48.710172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:48.808761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f679b9-fd86-4327-a1f2-ec603858dd37 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:48.852279 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:48.852997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:48.853333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:48.853659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:48.868892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:48.870343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:48.889090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:48.890174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:48.891064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:48.891064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:48.891064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:48.891064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 393/787] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:48.897007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.016401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba62b5-927f-4fa2-a5ef-4d104fb4fe48 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.059963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.078606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.079238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.079518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.079792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.080451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:49.080751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:49.082154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:49.099565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 395/788] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:49.104593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.187273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:49.207864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:49.215204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f43fe0-60cb-4f30-bae7-fdc715a41a0e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.254593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7e1e4-c1fd-4053-8ba9-71dd9ec888af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.255608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e1e4-c1fd-4053-8ba9-71dd9ec888af None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.256109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e1e4-c1fd-4053-8ba9-71dd9ec888af None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.256573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7e1e4-c1fd-4053-8ba9-71dd9ec888af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.257009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec459e11-0a4c-409b-8607-91db4faf2bb0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.268250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7e1e4-c1fd-4053-8ba9-71dd9ec888af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-cec7e1e4-c1fd-4053-8ba9-71dd9ec888af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:49.274567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:49.275491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:49.276322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:49.278776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 394/789] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:49.280378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c25b3b-d57b-4ba3-a22a-9e879d7227e2 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.280960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c25b3b-d57b-4ba3-a22a-9e879d7227e2 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.281375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c25b3b-d57b-4ba3-a22a-9e879d7227e2 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.287515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 395/790] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:49.288626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c25b3b-d57b-4ba3-a22a-9e879d7227e2 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.293856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bd03d-b622-4947-bcde-f7674c0ea016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.294597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd03d-b622-4947-bcde-f7674c0ea016 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.295020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd03d-b622-4947-bcde-f7674c0ea016 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.295387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd03d-b622-4947-bcde-f7674c0ea016 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.305522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bd03d-b622-4947-bcde-f7674c0ea016 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:49.307204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bd03d-b622-4947-bcde-f7674c0ea016 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6f6bd03d-b622-4947-bcde-f7674c0ea016 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:49.323174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:49.324351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:46:49.325610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:49.326356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 396/791] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:49.328418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d542d-bc67-420d-8889-a9e63fb3b178 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.328766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d542d-bc67-420d-8889-a9e63fb3b178 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.329152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d542d-bc67-420d-8889-a9e63fb3b178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.353564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f75d542d-bc67-420d-8889-a9e63fb3b178 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:49.354886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f75d542d-bc67-420d-8889-a9e63fb3b178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:49.411215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c25b3b-d57b-4ba3-a22a-9e879d7227e2 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.442451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 397/792] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:49.442451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d542d-bc67-420d-8889-a9e63fb3b178 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.477214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.477986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.478360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.478729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.479484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:49.479877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:49.481326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:49.483684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:49.488519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:49.498938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 396/793] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:49.504244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.543947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d542d-bc67-420d-8889-a9e63fb3b178 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.581352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.582314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.582798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.583421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.584414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:49.584890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:49.586554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:49.606113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 398/794] 10.222.0.72 () {56 vars in 1149 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:49.611510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.687613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a15674-67e3-475d-8f4b-9606edb4d98d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.740949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.741604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.741914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.742186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.753529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:49.755904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:49.771297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:49.772478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:49.773572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:49.773572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:49.773572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:49.777343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 397/795] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:49.778599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.790750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e337b8-b31c-40bf-8496-6d1244f4b27a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.793326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:49.828891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.829525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.829830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.830102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.836282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:49.842226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:49.843712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:49.861223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:49.865083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:49.866247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:49.866247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:49.866247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:49.866247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 399/796] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:49.866247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.909420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cecb8-9c46-4624-b12b-957ab1da6dd0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:49.956712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:49.958220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:49.959265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:49.959638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:49.974313 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:49.982110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:49.983045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 398/797] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:49.989354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:49.994131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf3619-000c-4970-9020-bed9c1d47f0d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.039500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.040239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.041019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.041356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.042047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:50.042397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.044414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.066352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 400/798] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:50.066352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.086565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.111312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061dff-34e5-4c68-b3e7-e37d6d0d3788 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.123498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.149636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.150711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.151677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.152173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.153178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:50.153706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.155392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.175995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 399/799] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:50.178409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.200766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5341a3-a1ba-4602-a2d2-ed2048627fde None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.247960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.248490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.248767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.249036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.249697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:50.250017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.252181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.268666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 401/800] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:50.268666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.292968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200677e-bb3b-4167-bd6b-e8899c9de733 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.330165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.330944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.331222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.331655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.338848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.340267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:50.351642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:50.352633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 400/801] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:50.353572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.361683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.386898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bbeca-3416-43af-b1c4-07f95a2b5841 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.396634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.425424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2889737-de7c-451d-95de-3ec6e128bf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.426655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2889737-de7c-451d-95de-3ec6e128bf4a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.426935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2889737-de7c-451d-95de-3ec6e128bf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.427198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2889737-de7c-451d-95de-3ec6e128bf4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.434792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2889737-de7c-451d-95de-3ec6e128bf4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.436215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2889737-de7c-451d-95de-3ec6e128bf4a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d2889737-de7c-451d-95de-3ec6e128bf4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:50.450864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:50.451782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:50.452691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 402/802] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:50.454045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef689246-259c-4ae9-b9b5-474a7123d2f8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.454045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef689246-259c-4ae9-b9b5-474a7123d2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.454045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef689246-259c-4ae9-b9b5-474a7123d2f8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.458481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 403/803] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:50.458686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef689246-259c-4ae9-b9b5-474a7123d2f8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.481426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7be7-6c10-45d2-a32f-85c5fa1cc460 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.520641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.521178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.521454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.521726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.533396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.534874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:50.552077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:50.553652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:50.554582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:50.554582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:50.554582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:50.554582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 401/804] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:50.554582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.585798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef689246-259c-4ae9-b9b5-474a7123d2f8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.625071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d660114-5a98-44c4-b666-6ed7ac89b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.625797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d660114-5a98-44c4-b666-6ed7ac89b409 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.626143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d660114-5a98-44c4-b666-6ed7ac89b409 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.626477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d660114-5a98-44c4-b666-6ed7ac89b409 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.627392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d660114-5a98-44c4-b666-6ed7ac89b409 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:50.627869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d660114-5a98-44c4-b666-6ed7ac89b409 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.629467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d660114-5a98-44c4-b666-6ed7ac89b409 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.644394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.647552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.657993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 404/805] 10.222.0.72 () {56 vars in 1135 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:50.659358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba215d-34f8-47f3-8359-1716d16e2f0a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.661151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba215d-34f8-47f3-8359-1716d16e2f0a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.661151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba215d-34f8-47f3-8359-1716d16e2f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.668324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b46c-b1b7-4fb5-a042-6c2aea4052b9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.679093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cbba215d-34f8-47f3-8359-1716d16e2f0a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:50.680394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cbba215d-34f8-47f3-8359-1716d16e2f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:50.705247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.705900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.706174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.706444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.707160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:50.707618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.709211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.729173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 402/806] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:50.729173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.769922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 405/807] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:50.771055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba215d-34f8-47f3-8359-1716d16e2f0a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.860254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2758e-127a-43f6-8d8d-5417009c1c4c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.888861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba215d-34f8-47f3-8359-1716d16e2f0a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:50.902156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.902763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.903045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.903318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.903962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:50.904267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.905764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.917429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 403/808] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:50.918243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:50.920908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.923657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:50.932599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d361e06-da1b-441e-b424-17e1c06693b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:50.938310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d361e06-da1b-441e-b424-17e1c06693b7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.938310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d361e06-da1b-441e-b424-17e1c06693b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.938310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d361e06-da1b-441e-b424-17e1c06693b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.944036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d361e06-da1b-441e-b424-17e1c06693b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:50.945459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d361e06-da1b-441e-b424-17e1c06693b7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-7d361e06-da1b-441e-b424-17e1c06693b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:50.960475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:50.961587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:50.962578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:50.962578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:50.962578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:50.962578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 406/809] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:50.963131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71261427-8226-454f-94c6-7c495c76353b None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:50.963236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71261427-8226-454f-94c6-7c495c76353b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:50.963437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71261427-8226-454f-94c6-7c495c76353b None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:50.969249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 407/810] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:50.969249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71261427-8226-454f-94c6-7c495c76353b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.028561 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a63415-9ca9-48f8-821e-b8ba72d8599c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.063993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbd178-5c24-4ace-aaf3-dd7dc18ba536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.064954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd178-5c24-4ace-aaf3-dd7dc18ba536 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.065231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd178-5c24-4ace-aaf3-dd7dc18ba536 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.065503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd178-5c24-4ace-aaf3-dd7dc18ba536 None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/e658441f5ed34cfbb926364644fbaed6/roles/fd66e13437394d4f98a74978756fb6e0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.073306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71261427-8226-454f-94c6-7c495c76353b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.079830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd178-5c24-4ace-aaf3-dd7dc18ba536 None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=e658441f5ed34cfbb926364644fbaed6, role_id=fd66e13437394d4f98a74978756fb6e0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.081257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd178-5c24-4ace-aaf3-dd7dc18ba536 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.106753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.107368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.107643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.107912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.115237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.116679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:51.127513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:51.128457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:51.129399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 408/811] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:46:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:51.133367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.156634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 404/812] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:50 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/e658441f5ed34cfbb926364644fbaed6/roles/fd66e13437394d4f98a74978756fb6e0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:51.157687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea502dd-3e50-4e08-8e4a-e82d963cbadc None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.158138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea502dd-3e50-4e08-8e4a-e82d963cbadc None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.158274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea502dd-3e50-4e08-8e4a-e82d963cbadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.175400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:51.180061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1ea502dd-3e50-4e08-8e4a-e82d963cbadc None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:51.181401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1ea502dd-3e50-4e08-8e4a-e82d963cbadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:51.234504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:51.247605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fcc88-aa1f-4f16-91dd-675ada8a73e7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.265766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 405/813] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:51.265766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea502dd-3e50-4e08-8e4a-e82d963cbadc None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.298579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.299110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.299387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.299657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.301587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:51.301932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.303332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.318824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 409/814] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:51.319786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.381468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea502dd-3e50-4e08-8e4a-e82d963cbadc None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.418684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.419320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.419685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.419962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.420552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.421954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:51.431386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:51.432438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:51.433974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 406/815] 10.222.0.72 () {56 vars in 1129 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:51.434793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.458465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:51.465268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2df00-71f6-4143-81d8-dbd82c6451e3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.509296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:51.533219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.535272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.535915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.536389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.537675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:51.538228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.541497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.574034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 410/816] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:51.576299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.601407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcf133-8cc7-43cb-9c69-96ba1bec6818 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.637191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.637836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.638114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.638637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.648052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.649512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:51.660748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:51.663925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 407/817] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:51.664843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.702338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525df1-6c4e-471f-8a10-7845b15f1752 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.739790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.740400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.740676 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.740942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.741563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:51.741909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.743373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.752961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 411/818] 10.222.0.72 () {56 vars in 1124 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:51.753846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.766806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fae6c6-358a-403d-b2dd-86e77668f9f8 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.773854 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:51.777558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:51.800941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.802129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.802437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.802685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.803236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.804611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:51.814273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:51.815498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 12:46:51.816580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:51.821113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 408/819] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:51.822041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:51.865446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21e4d7-7e5c-4b68-9662-cbc39a53d593 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.904403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.905015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.905299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.905770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.928991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20942c-01f5-4ae9-89c7-c281801125d3 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:51.965273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:51.965891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:51.966170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:51.966442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:51.967134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:51.967436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.968857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.978648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:51.980080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:51.987178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 409/820] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:51.987178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.007602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:52.029287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:52.092638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 412/821] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:46:52.094346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.107031 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb3398-bb5f-4305-86d6-1d1f8c7f1e1f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.143268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:52.144344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:52.145655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:52.145945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:52.146375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:52.146680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:52.148053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:52.157717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 410/822] 10.222.0.72 () {56 vars in 1124 bytes} [Tue Sep 13 12:46:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:52.157717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.204015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d0dba-a030-4027-ad53-305a95c57460 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.224754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:52.239640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:52.240255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:52.240682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:52.240953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:52.241581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] RBAC: Adding query filter params (name=network) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:52.242013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:52.243339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:52.247798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 413/823] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:46:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:52.248451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.260605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:52.271092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb68c2-5174-449e-89a2-bd59f5f0076d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.311981 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:52.312685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:52.312965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:52.313242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/a08e9745841d4716a64700da59eebf04/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:52.315231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=a08e9745841d4716a64700da59eebf04, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:52.316652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:52.354906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1fdb8-258c-47e7-a4d6-70099b3d3cbd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.389775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 411/824] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:46:52 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/a08e9745841d4716a64700da59eebf04/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:46:52.389775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.399051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:52.399677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:52.400014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:52.400347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:52.414504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:52.416167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:52.433175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:52.439285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:52.440385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:52.440385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:52.440385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:52.440385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 414/825] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:52.440385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.494897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:52.543281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc81263-c376-4a38-be27-48bd86f57a3c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.587409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5eb689-41ba-4bc0-900d-97e992f57ba9 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.588932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:52.589545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:52.590471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:52.590471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:52.591435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:52.592827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:52.624846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:52.625513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:52.625866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:52.626127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:52.626746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:52.627046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:52.628395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:52.638421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 415/826] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:46:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:52.643067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.707990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 412/827] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:46:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:52.714922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.743698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:52.764454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:52.768617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c575f87-5ebd-4590-aa53-e3d10e7b51f8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.827378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:52.828704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:52.829125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:52.829644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:52.830610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:52.831049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:52.832746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:52.860632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 416/828] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:52.867092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:52.962694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c24c-0f01-474b-82b5-b2362906a2c9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:52.985555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c3b5f-bca1-45bc-8211-8f59e1d9f5c8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:53.003094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:53.003851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:53.004758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:53.005382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:53.006504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:53.007860 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:53.025048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:53.026011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:53.027205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:53.027205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:53.034512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:53.036583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:53.054249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:53.055322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:53.056240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:53.057597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 417/829] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:53.063397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:53.087974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:53.113226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 413/830] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:46:52 2022] POST /identity/v3/endpoints => generated 337 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:46:53.175938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0560b1f-d414-436a-a4bd-acbe4276e991 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:53.220038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da61b50-ae35-403c-a40d-7815f0a7e7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:53.220574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61b50-ae35-403c-a40d-7815f0a7e7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:53.220852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61b50-ae35-403c-a40d-7815f0a7e7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:53.221121 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da61b50-ae35-403c-a40d-7815f0a7e7f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:53.221849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da61b50-ae35-403c-a40d-7815f0a7e7f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:53.222153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da61b50-ae35-403c-a40d-7815f0a7e7f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:53.224370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da61b50-ae35-403c-a40d-7815f0a7e7f5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:53.238027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 418/831] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:53.244013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:53.357055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:53.401983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a358ae3-c03c-4836-ae23-1ae999ea3525 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:53.402333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a358ae3-c03c-4836-ae23-1ae999ea3525 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:53.402554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a358ae3-c03c-4836-ae23-1ae999ea3525 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:53.405889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 419/832] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:53.410786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6accc-b4dc-481a-b68c-52f15883d387 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:53.410976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66974c-be5d-4333-b954-eacaeea73d79 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:53.411057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6accc-b4dc-481a-b68c-52f15883d387 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:53.411343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6accc-b4dc-481a-b68c-52f15883d387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:53.443493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f5d6accc-b4dc-481a-b68c-52f15883d387 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:53.444763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f5d6accc-b4dc-481a-b68c-52f15883d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:53.472700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:53.473808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:53.474087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:53.474408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:53.475134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=3b21d991d02b4f95a8d0fcec0a334495) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:53.475937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:53.477675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:53.485675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 414/833] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:53 2022] GET /identity/v3/role_assignments?user.id=3b21d991d02b4f95a8d0fcec0a334495&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:53.536912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 420/834] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:46:53.817261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:53.920257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86439ee-b335-4cfd-985a-885fc84a15c8 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:53.952286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:53.958736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:53.959374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:53.959652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:53.959925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:53.971064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:53.977075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:53.977996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:46:53.978833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:53.983024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 415/835] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:53.984366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6accc-b4dc-481a-b68c-52f15883d387 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:53.986750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:54.086756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6accc-b4dc-481a-b68c-52f15883d387 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:54.121860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:54.122476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:54.122759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:54.123034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:54.123704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:54.124006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:54.125370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:54.137632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 421/836] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:46:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:54.140164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:54.242093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67569d-a298-4025-b33f-c0eec0116305 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:54.277383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:54.278055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:54.278332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:54.278604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:54.290162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:54.291614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:54.306728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:54.307682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:54.308564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:54.308564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:54.308564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:54.308564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 416/837] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:54.313668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:54.380509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:54.419111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50297ac-eda6-49e1-9e84-555b1b89a864 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:54.453379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:54.454017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:54.454391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:54.454665 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:54.455304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:54.455604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:54.456948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:54.471635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 422/838] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:54.477497 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:54.529367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:54.590396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7418c8-793e-41dc-86f3-2425f54adb2e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:54.625248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de26b2-e398-47a6-9b9b-61e62a0c5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:54.626245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de26b2-e398-47a6-9b9b-61e62a0c5170 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:54.626795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de26b2-e398-47a6-9b9b-61e62a0c5170 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:54.627259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de26b2-e398-47a6-9b9b-61e62a0c5170 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:54.648260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de26b2-e398-47a6-9b9b-61e62a0c5170 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:54.650124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de26b2-e398-47a6-9b9b-61e62a0c5170 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-43de26b2-e398-47a6-9b9b-61e62a0c5170 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:54.667420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:54.668417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:54.669348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 417/839] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:46:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:54.674831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:54.781156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c6f8f-e5aa-4c4a-811f-49f7dda7857a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:54.806103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:54.810119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb10dad-940d-4d7b-a551-28dffbb5ef73 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:54.810406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb10dad-940d-4d7b-a551-28dffbb5ef73 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:54.810685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb10dad-940d-4d7b-a551-28dffbb5ef73 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:54.812012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 418/840] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:54.817772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:54.818082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5779-8506-4cd5-b77f-538bff2a23b6 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:54.818504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5779-8506-4cd5-b77f-538bff2a23b6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:54.818578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:54.818648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b5779-8506-4cd5-b77f-538bff2a23b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:54.820958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:54.820958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:54.820958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:54.820958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:54.821780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:54.842565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-730b5779-8506-4cd5-b77f-538bff2a23b6 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:54.844123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-730b5779-8506-4cd5-b77f-538bff2a23b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:54.846848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 423/841] 10.222.0.72 () {56 vars in 1149 bytes} [Tue Sep 13 12:46:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:54.854737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:54.929261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 419/842] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:54.975533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd56e2-ad24-4ad9-9e36-74b57e222ec1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.014165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.015107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.015600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.016062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.068564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:55.088823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:55.090603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:55.190955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 424/843] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:46:55.199632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b5779-8506-4cd5-b77f-538bff2a23b6 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.225034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.245995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:55.316532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:55.340002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8def-b32c-4582-aa9f-24527e4ffde7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.340190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b5779-8506-4cd5-b77f-538bff2a23b6 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.375480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfc931-8f6c-4a62-a9f1-a03a1d986f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.376164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc931-8f6c-4a62-a9f1-a03a1d986f86 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.376397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc931-8f6c-4a62-a9f1-a03a1d986f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.376674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfc931-8f6c-4a62-a9f1-a03a1d986f86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.380818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8d969-c32e-42a2-82f8-263bbb3aca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.381703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8d969-c32e-42a2-82f8-263bbb3aca00 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.382322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8d969-c32e-42a2-82f8-263bbb3aca00 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.382852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8d969-c32e-42a2-82f8-263bbb3aca00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.387907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfc931-8f6c-4a62-a9f1-a03a1d986f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e2dfc931-8f6c-4a62-a9f1-a03a1d986f86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:55.392691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:55.393846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:55.394749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 425/844] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:55.395751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef59788-f531-4c9a-b5af-cd9ec0b49563 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.395751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef59788-f531-4c9a-b5af-cd9ec0b49563 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.395751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef59788-f531-4c9a-b5af-cd9ec0b49563 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.397770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 426/845] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:55.399704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef59788-f531-4c9a-b5af-cd9ec0b49563 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.402584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8d969-c32e-42a2-82f8-263bbb3aca00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:55.404383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8d969-c32e-42a2-82f8-263bbb3aca00 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-4fc8d969-c32e-42a2-82f8-263bbb3aca00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:55.423821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:55.424822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:55.425773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:55.425773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:55.425773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:55.425773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 420/846] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:55.427618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9838f-5db6-4d75-86f7-b16dfa92be6d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.427618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9838f-5db6-4d75-86f7-b16dfa92be6d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.427618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9838f-5db6-4d75-86f7-b16dfa92be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.445249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-a7e9838f-5db6-4d75-86f7-b16dfa92be6d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:55.447092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a7e9838f-5db6-4d75-86f7-b16dfa92be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:55.517413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef59788-f531-4c9a-b5af-cd9ec0b49563 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.521448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 421/847] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:55.522148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9838f-5db6-4d75-86f7-b16dfa92be6d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.538409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:55.563930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.564528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.564809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.565077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.565759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:55.566061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:55.568117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:55.585803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 427/848] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:55.586777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.591450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:55.656935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9838f-5db6-4d75-86f7-b16dfa92be6d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.693208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.693890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.694169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.694531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.695215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:55.695577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:55.697044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:55.715288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 422/849] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:55.716353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.718614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079473d6-7ce8-4253-b6ee-beed94aac427 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.764474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.765637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.766671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.767273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.781508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:55.783833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:55.796731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:55.803220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:55.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:55.805096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:55.805096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:55.805096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:55.805721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 428/850] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:55.806046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.828632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020447b8-bf20-4d25-a1b3-3ca3a6ef0714 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.862385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.863010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.863289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.864289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.864408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:55.871156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:55.872596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:55.884584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:55.885592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 423/851] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:55.886535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.915416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7b6a6-dade-4b9a-90c6-c445594c44be None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:55.948431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5123fb-f1a1-4d65-a489-83d0b7bd56c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:55.949058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5123fb-f1a1-4d65-a489-83d0b7bd56c9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.949338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5123fb-f1a1-4d65-a489-83d0b7bd56c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.949609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5123fb-f1a1-4d65-a489-83d0b7bd56c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.950488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5123fb-f1a1-4d65-a489-83d0b7bd56c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:55.951842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5123fb-f1a1-4d65-a489-83d0b7bd56c9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-0a5123fb-f1a1-4d65-a489-83d0b7bd56c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:55.962377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:55.965830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:55.966742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 429/852] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:55.967527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417ad09-fd08-46fb-9da2-2b3a1bbc2998 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:55.967527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417ad09-fd08-46fb-9da2-2b3a1bbc2998 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:55.967527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417ad09-fd08-46fb-9da2-2b3a1bbc2998 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:55.969980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 430/853] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:55.971209 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417ad09-fd08-46fb-9da2-2b3a1bbc2998 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:55.985137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740c593-44f3-42e4-8cae-49d8be2744a7 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.018226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.018818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.019100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.019363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.020141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:56.020436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.021831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.023186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.038710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 424/854] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:56.038710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.074331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417ad09-fd08-46fb-9da2-2b3a1bbc2998 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.094956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.108802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b6df4-e599-40fd-9366-d23ba98d8173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.109400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6df4-e599-40fd-9366-d23ba98d8173 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.109683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6df4-e599-40fd-9366-d23ba98d8173 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.109987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6df4-e599-40fd-9366-d23ba98d8173 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.114100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b6df4-e599-40fd-9366-d23ba98d8173 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:56.114100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b6df4-e599-40fd-9366-d23ba98d8173 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.114100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b6df4-e599-40fd-9366-d23ba98d8173 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.123859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 431/855] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:56.125019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b2a16-c5e3-4bfc-a08d-0535e214e034 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.125310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b2a16-c5e3-4bfc-a08d-0535e214e034 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.125583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b2a16-c5e3-4bfc-a08d-0535e214e034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.131283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4e74e-f2cd-4546-9434-81ede008f635 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.143708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9e2b2a16-c5e3-4bfc-a08d-0535e214e034 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:56.145398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9e2b2a16-c5e3-4bfc-a08d-0535e214e034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:56.170060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.170662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.170937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.171202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.171821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:56.172118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.173463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.177888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 425/856] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:56.178654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.215832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 432/857] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:56.216931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b2a16-c5e3-4bfc-a08d-0535e214e034 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.250008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.291331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a86d9-d3d1-4b85-ae9d-d530591be5b8 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.320942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b2a16-c5e3-4bfc-a08d-0535e214e034 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.330437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.331046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.331324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.331591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.333385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.339529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.340953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:46:56.353690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:56.354844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 12:46:56.355974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:56.359643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 426/858] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:56.360376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.360499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.361018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.361300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.361576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.362346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=e658441f5ed34cfbb926364644fbaed6) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:56.362656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.365305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.377766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 433/859] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/role_assignments?user.id=e658441f5ed34cfbb926364644fbaed6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:56.377766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.486722 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864b20e-9bd1-42fb-864d-93d69a4ec4f3 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.498609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d9d72-dd64-4d49-82b3-bf420209cc78 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.506243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.526579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.527307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.527650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.527931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.528724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:56.529139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.530563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.537853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 427/860] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:56.538048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.538180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.538257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.538257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.538257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.538838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:56.539169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.540647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.570365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 434/861] 10.222.0.72 () {56 vars in 1135 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:56.576966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.636929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.697347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e620073-3969-47fe-a15e-d9aa072a519b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.740237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e69a15-0a1c-4b1d-b513-144d2bde00de None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.746564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.746990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.747266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.747533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.748110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.749947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:56.761396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:56.762444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:56.763327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 428/862] 10.222.0.72 () {56 vars in 1132 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:56.771234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.777951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.778567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.778848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.779120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.833290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.861894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.864447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.903711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:56.906848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318d0c1-8ce8-4073-b9c6-4733b52c5f91 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:56.952639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:56.953254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:56.953530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:56.953823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:56.954451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:56.954747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:56.956112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:56.967297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 429/863] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:56.968768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:56.996905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 435/864] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 12:46:57.002342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:57.121964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09252ab4-8cd5-4205-a2db-9d5fbeee7c8a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:57.155955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:57.163239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:57.164418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:57.164961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:57.165563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:57.179892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:57.181637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:57.200994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:57.202034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:57.202933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:57.202933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:57.202933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:57.205878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 436/865] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:57.209295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:57.220487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd56e7a-9097-4f89-8b32-f4a2dcfd5fd9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:57.234909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:57.264744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:57.265278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:57.266017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:57.266537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:57.267600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:57.267906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:57.269563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:57.282513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 430/866] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:46:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:57.334862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a46b0-5750-4706-b026-29c5cfdaa79c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:57.373388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:57.374039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:57.374321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:57.374595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:57.375255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:57.375560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:57.376955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:57.391217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 437/867] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:57.397599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:57.515867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05863afa-d930-4d65-975a-ea971d2c31d0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:57.554965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:57.555600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:57.555926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:57.556202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:57.563090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:57.564516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:46:57.576585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:57.577485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:57.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 431/868] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:57.584727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:57.633982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:57.689474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc26a1-8435-4afa-b650-17b3d68ebab0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:57.724701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806e00d-86b3-4e33-8753-4ac9e342a9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:57.725312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e00d-86b3-4e33-8753-4ac9e342a9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:57.725593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e00d-86b3-4e33-8753-4ac9e342a9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:57.725917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e00d-86b3-4e33-8753-4ac9e342a9f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:57.726615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806e00d-86b3-4e33-8753-4ac9e342a9f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:57.726923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806e00d-86b3-4e33-8753-4ac9e342a9f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:57.728344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806e00d-86b3-4e33-8753-4ac9e342a9f1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:57.742740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 438/869] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:57.748600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:57.785484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:57.854241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d99e4b-da39-4a21-ba48-f061d7bd32c7 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:57.890250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3e7a7-6264-4a04-860e-4daf79bca9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:57.890909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3e7a7-6264-4a04-860e-4daf79bca9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:57.891194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3e7a7-6264-4a04-860e-4daf79bca9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:57.891470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3e7a7-6264-4a04-860e-4daf79bca9a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:57.892248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3e7a7-6264-4a04-860e-4daf79bca9a8 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=a08e9745841d4716a64700da59eebf04) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:57.892559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3e7a7-6264-4a04-860e-4daf79bca9a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:57.894011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3e7a7-6264-4a04-860e-4daf79bca9a8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:57.900474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 432/870] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:46:57 2022] GET /identity/v3/role_assignments?user.id=a08e9745841d4716a64700da59eebf04&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:58.745629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51303440-491e-4064-bf70-9881b3dbffa1 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:58.746217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51303440-491e-4064-bf70-9881b3dbffa1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:58.746217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51303440-491e-4064-bf70-9881b3dbffa1 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:58.747752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 439/871] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:58.753139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adfeb8-fec5-4e2a-8010-b25d577420d3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:58.753428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adfeb8-fec5-4e2a-8010-b25d577420d3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:58.753702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adfeb8-fec5-4e2a-8010-b25d577420d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:58.772598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-32adfeb8-fec5-4e2a-8010-b25d577420d3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:58.773781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-32adfeb8-fec5-4e2a-8010-b25d577420d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:58.851522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 433/872] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:58.946504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998da2-867f-43de-976f-a8df9b3d3253 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:58.946775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998da2-867f-43de-976f-a8df9b3d3253 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:58.946966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998da2-867f-43de-976f-a8df9b3d3253 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:58.948536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 440/873] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:58.955419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd11c2-583d-4156-9a4c-2bf3f13a909a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:58.955880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd11c2-583d-4156-9a4c-2bf3f13a909a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:58.955965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd11c2-583d-4156-9a4c-2bf3f13a909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:58.974642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4acd11c2-583d-4156-9a4c-2bf3f13a909a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:58.975822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4acd11c2-583d-4156-9a4c-2bf3f13a909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:58.998258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:59.046401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 434/874] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:59.136722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e998da2-867f-43de-976f-a8df9b3d3253 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:59.241451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e998da2-867f-43de-976f-a8df9b3d3253 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:59.243851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:59.276070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:59.276705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:59.276985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:59.277309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:59.289573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:46:59.294417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:59.295405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:46:59.296301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 441/875] 10.222.0.72 () {56 vars in 1120 bytes} [Tue Sep 13 12:46:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:46:59.302314 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd11c2-583d-4156-9a4c-2bf3f13a909a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:59.327350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:59.423695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd11c2-583d-4156-9a4c-2bf3f13a909a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:59.446468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c64a9-3919-46ea-8a9e-26152cbf2b92 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:59.478109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:59.478956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:59.479537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:59.479815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:59.481084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:59.481521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:59.483648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:59.491775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828133c5-0e95-41b4-a1c8-4e681d965aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:59.492314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828133c5-0e95-41b4-a1c8-4e681d965aa2 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:59.492588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828133c5-0e95-41b4-a1c8-4e681d965aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:59.492971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828133c5-0e95-41b4-a1c8-4e681d965aa2 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:59.493623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828133c5-0e95-41b4-a1c8-4e681d965aa2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:59.494546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 435/876] 10.222.0.72 () {56 vars in 1115 bytes} [Tue Sep 13 12:46:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:46:59.495101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828133c5-0e95-41b4-a1c8-4e681d965aa2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:59.503019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:59.524487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:59.596928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 442/877] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:46:59 2022] POST /identity/v3/services => generated 251 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:46:59.599206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01623b19-6dd7-4bf4-8b88-df66e9511b15 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:59.599703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01623b19-6dd7-4bf4-8b88-df66e9511b15 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:59.600190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01623b19-6dd7-4bf4-8b88-df66e9511b15 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:59.602072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 443/878] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:46:59.606783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3beaaec-a5ec-47db-9b89-ee1c630dff2c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:59.607922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96954ee6-1c32-4d70-944c-cb03f3903dc2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:59.608417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96954ee6-1c32-4d70-944c-cb03f3903dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:59.608941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96954ee6-1c32-4d70-944c-cb03f3903dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:59.629570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-96954ee6-1c32-4d70-944c-cb03f3903dc2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:46:59.631302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-96954ee6-1c32-4d70-944c-cb03f3903dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:46:59.653019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:59.656343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:59.657147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:59.657495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:59.657864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:59.670744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:59.672646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:46:59.690255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:46:59.691288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:46:59.693353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:46:59.693353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:46:59.693353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:46:59.693353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 436/879] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:46:59.697278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:59.720092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 444/880] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:46:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:46:59.785493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:46:59.823278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dfcad-c1f5-4232-829b-6c86d36d2d09 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:59.860544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:46:59.878245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:46:59.878245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:46:59.878245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:46:59.878245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:46:59.878245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:46:59.878245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:46:59.878708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 437/881] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:46:59.882646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96954ee6-1c32-4d70-944c-cb03f3903dc2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:46:59.988242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96954ee6-1c32-4d70-944c-cb03f3903dc2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:46:59.992429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:00.031702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:00.032511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.032635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.032873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.041098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:00.042543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:47:00.055684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:00.056701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:00.058031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 445/882] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:46:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:00.062226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:00.091353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:00.136510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf774b46-2923-4c06-92d6-775c0825a014 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:00.170216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:00.175701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336be740-b50c-472f-91a6-e4990e28920a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:00.180697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55410631-a9c0-4ae1-8725-9d8033c8f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:00.181501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55410631-a9c0-4ae1-8725-9d8033c8f2c1 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.181951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55410631-a9c0-4ae1-8725-9d8033c8f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.184454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55410631-a9c0-4ae1-8725-9d8033c8f2c1 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.184454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55410631-a9c0-4ae1-8725-9d8033c8f2c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:00.184858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55410631-a9c0-4ae1-8725-9d8033c8f2c1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:00.214450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd6de9-dbae-4a04-ab99-24069da8056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:00.215056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd6de9-dbae-4a04-ab99-24069da8056a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.215337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd6de9-dbae-4a04-ab99-24069da8056a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.215615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd6de9-dbae-4a04-ab99-24069da8056a None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.216381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd6de9-dbae-4a04-ab99-24069da8056a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:00.216687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd6de9-dbae-4a04-ab99-24069da8056a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:00.218135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd6de9-dbae-4a04-ab99-24069da8056a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:00.236229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 446/883] 10.222.0.72 () {56 vars in 1149 bytes} [Tue Sep 13 12:47:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:00.237417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0d1f-f394-44aa-8286-93f76689122a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.237703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0d1f-f394-44aa-8286-93f76689122a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.238007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0d1f-f394-44aa-8286-93f76689122a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.240011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 447/884] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:00.243763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c0d1f-f394-44aa-8286-93f76689122a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:00.281822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 438/885] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:46:59 2022] POST /identity/v3/services => generated 269 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:47:00.281822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a86c75-0767-4147-9fc8-a75b4fde6501 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.281822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a86c75-0767-4147-9fc8-a75b4fde6501 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.281822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a86c75-0767-4147-9fc8-a75b4fde6501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.310840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f7a86c75-0767-4147-9fc8-a75b4fde6501 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:00.312579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f7a86c75-0767-4147-9fc8-a75b4fde6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:00.326776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:00.369404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c0d1f-f394-44aa-8286-93f76689122a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:00.408518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:00.409148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.409416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.409687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.422827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 439/886] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:00.487449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:00.489515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:00.557800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:00.595041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 448/887] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:00 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 12:47:00.600772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a86c75-0767-4147-9fc8-a75b4fde6501 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:00.695090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:00.713420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a86c75-0767-4147-9fc8-a75b4fde6501 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:00.737892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:00.762159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:00.762798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.763079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.763353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.777987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:00.780159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:00.800751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f5856-4974-46d6-9f70-8659f57d5923 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:00.801758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:00.802661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:00.803534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:00.803534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:00.803534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:00.804856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 440/888] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:00.809578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:00.849983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:00.850908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.851419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.851998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.852826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:00.854426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:00.864981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:00.866381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:00.867238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:00.874605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 449/889] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:00.875694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:00.910326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:00.941299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923e56d-b2a2-48c4-b91e-a56a624008c4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:00.989998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:00.990656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:00.990936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:00.991206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:00.992105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:00.992405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:00.994565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:01.000176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a06caf-f4ef-4976-b16e-53c039035e93 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:01.010036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 441/890] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:01.015140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:01.045714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:01.046627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:01.048609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:01.049158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:01.049597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:01.050397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:01.050900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:01.052660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:01.062560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 450/891] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:01.065168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:01.121707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417093c2-c247-4ea6-b773-3cef0954fa30 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:01.157094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:01.157985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:01.158420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:01.158778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:01.166234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:01.167770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:01.179103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:47:01.180332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:01.181407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:01.182603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:01.187603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f38f4-f5f6-44e6-8b0d-6b2e3cdaebb6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:01.193475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 442/892] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:01.193475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:01.229135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:01.229759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:01.230048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:01.230322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:01.230957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:01.231260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:01.232614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:01.245847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 451/893] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:01.296825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:01.307318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174c64-e551-4908-8d99-03b7adbd319b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:01.348670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea11d7-7e2b-4d69-be7b-ce9d5a79c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:01.349300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea11d7-7e2b-4d69-be7b-ce9d5a79c020 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:01.349574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea11d7-7e2b-4d69-be7b-ce9d5a79c020 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:01.349884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea11d7-7e2b-4d69-be7b-ce9d5a79c020 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:01.350664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea11d7-7e2b-4d69-be7b-ce9d5a79c020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:01.350971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea11d7-7e2b-4d69-be7b-ce9d5a79c020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:01.352416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea11d7-7e2b-4d69-be7b-ce9d5a79c020 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:01.369889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 443/894] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:01.372471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:01.489682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b74eb6-00bc-4edc-8b60-4aeb7dcc05ce None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:01.525598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89481a9f-5412-45d0-9aff-061c54bfd6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:01.526238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89481a9f-5412-45d0-9aff-061c54bfd6dc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:01.526514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89481a9f-5412-45d0-9aff-061c54bfd6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:01.526781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89481a9f-5412-45d0-9aff-061c54bfd6dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:01.527499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89481a9f-5412-45d0-9aff-061c54bfd6dc None admin] RBAC: Adding query filter params (role.id=84a9de490add48a7b98e6e4a899e0f30, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=e658441f5ed34cfbb926364644fbaed6) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:01.527799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89481a9f-5412-45d0-9aff-061c54bfd6dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:01.529202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89481a9f-5412-45d0-9aff-061c54bfd6dc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:01.541903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 452/895] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:01 2022] GET /identity/v3/role_assignments?user.id=e658441f5ed34cfbb926364644fbaed6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=84a9de490add48a7b98e6e4a899e0f30 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:01.961000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6ce1-4973-4b42-a92a-6824bef510a6 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:01.961000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6ce1-4973-4b42-a92a-6824bef510a6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:01.961000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb6ce1-4973-4b42-a92a-6824bef510a6 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:01.961903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 444/896] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:01.967359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2c96f-90b0-4eb8-9aff-8016a7567392 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:01.968219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2c96f-90b0-4eb8-9aff-8016a7567392 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:01.968693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2c96f-90b0-4eb8-9aff-8016a7567392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:01.987603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-44f2c96f-90b0-4eb8-9aff-8016a7567392 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:01.989889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-44f2c96f-90b0-4eb8-9aff-8016a7567392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:02.009267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:02.061199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 453/897] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:02.331428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6ce1-4973-4b42-a92a-6824bef510a6 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:02.437927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb6ce1-4973-4b42-a92a-6824bef510a6 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:02.477962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e33263-5b08-4125-8b4a-4a0cbe64079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:02.478610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33263-5b08-4125-8b4a-4a0cbe64079b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:02.478885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33263-5b08-4125-8b4a-4a0cbe64079b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:02.479152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33263-5b08-4125-8b4a-4a0cbe64079b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:02.479741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e33263-5b08-4125-8b4a-4a0cbe64079b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:02.481128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e33263-5b08-4125-8b4a-4a0cbe64079b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-55e33263-5b08-4125-8b4a-4a0cbe64079b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:02.491335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:02.492396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:02.493250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 445/898] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:47:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:02.500361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2c96f-90b0-4eb8-9aff-8016a7567392 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:02.575552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce6c2-a307-4435-a33c-42cc860bc959 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:02.575904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce6c2-a307-4435-a33c-42cc860bc959 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:02.576106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ce6c2-a307-4435-a33c-42cc860bc959 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:02.577685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 446/899] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:02.583586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97878e9-56e6-42a6-9f26-dff8b7bdd630 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:02.583820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97878e9-56e6-42a6-9f26-dff8b7bdd630 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:02.584082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97878e9-56e6-42a6-9f26-dff8b7bdd630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:02.605194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-b97878e9-56e6-42a6-9f26-dff8b7bdd630 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:02.605728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2c96f-90b0-4eb8-9aff-8016a7567392 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:02.606433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b97878e9-56e6-42a6-9f26-dff8b7bdd630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:02.619621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:02.646909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:02.647815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:02.648292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:02.648740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:02.649524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:02.650111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:02.651617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:02.658195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 454/900] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:47:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:02.669778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:02.687509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:02.696539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 447/901] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:02.772839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7552ec-9c9d-41e9-88b2-58ddfaca956f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:02.808424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:02.809393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:02.809924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:02.810421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:02.811250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:02.811760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:02.813300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:02.819396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 455/902] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:47:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:02.825157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97878e9-56e6-42a6-9f26-dff8b7bdd630 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:02.927639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97878e9-56e6-42a6-9f26-dff8b7bdd630 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:02.963602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:02.964575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:02.965071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:02.965556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:02.966473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] RBAC: Adding query filter params (interface=public, service_id=72884c1431534eabbd612f3f6e8056eb, region_id=RegionOne) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:02.966980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:02.968532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:02.977369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 448/903] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:47:02 2022] GET /identity/v3/endpoints?service_id=72884c1431534eabbd612f3f6e8056eb&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:02.979938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:02.982521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:02.994064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:03.078302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:03.091974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de5ebb-99a5-4272-80af-9fcc0132c448 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:03.104255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8de0d4-7269-458f-9158-b6ca3515b89f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:03.131319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:03.131978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.132398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.132742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.133413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:03.135465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:03.139761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:03.140593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.141065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.141520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.142349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:03.143871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:03.146425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:03.147374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:03.148270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 456/904] 10.222.0.72 () {56 vars in 1132 bytes} [Tue Sep 13 12:47:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:03.155027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:03.158758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 449/905] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:47:02 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 12:47:03.276194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6023f26-8439-47d1-af2d-f40cb6d061c1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:03.314443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:03.315501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.316028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.317055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.318055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:03.318666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:03.320295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:03.326560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 457/906] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:03.333371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:03.410833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:03.443025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa7dd-6838-4807-b22d-ce52f00aeb9a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:03.483457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deba5b-31b1-40d5-b2a5-8062ccf8ff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:03.484123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deba5b-31b1-40d5-b2a5-8062ccf8ff98 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.484404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deba5b-31b1-40d5-b2a5-8062ccf8ff98 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.484675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deba5b-31b1-40d5-b2a5-8062ccf8ff98 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.485316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deba5b-31b1-40d5-b2a5-8062ccf8ff98 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:03.485618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deba5b-31b1-40d5-b2a5-8062ccf8ff98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:03.487025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deba5b-31b1-40d5-b2a5-8062ccf8ff98 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:03.493404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 450/907] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:03.499409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:03.513043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:03.572858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fb133-5c7c-476c-b111-91f18a1007bc None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.573049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fb133-5c7c-476c-b111-91f18a1007bc None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.573311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fb133-5c7c-476c-b111-91f18a1007bc None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.574666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 451/908] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:03.580142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3a57a-c564-457b-8d6a-06a50f71d2c3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.580421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3a57a-c564-457b-8d6a-06a50f71d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.580696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3a57a-c564-457b-8d6a-06a50f71d2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.602759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f0f3a57a-c564-457b-8d6a-06a50f71d2c3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:03.603957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f0f3a57a-c564-457b-8d6a-06a50f71d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:03.612732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab3868-d32c-46fd-8669-42fd7f4617af None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:03.664952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:03.665784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.666147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.666536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.667517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] RBAC: Adding query filter params (interface=public, service_id=5ffd468d508e4956aa23e248e0b243a9, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:03.667824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:03.669435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:03.677149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 458/909] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:47:03 2022] GET /identity/v3/endpoints?service_id=5ffd468d508e4956aa23e248e0b243a9&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:03.682633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:03.705781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 452/910] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:03.762733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b15e-0c29-4635-8f68-7d6d1d806ded None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.763838 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b15e-0c29-4635-8f68-7d6d1d806ded None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.764124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96b15e-0c29-4635-8f68-7d6d1d806ded None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.765730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 453/911] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:03.770398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:03.772346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f79d-f4c0-4a44-8d3c-b4ce6eab7ae7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.772630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f79d-f4c0-4a44-8d3c-b4ce6eab7ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.772901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8f79d-f4c0-4a44-8d3c-b4ce6eab7ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.795537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-88b8f79d-f4c0-4a44-8d3c-b4ce6eab7ae7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:03.797285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-88b8f79d-f4c0-4a44-8d3c-b4ce6eab7ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:03.798881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1562159-76b2-4dd0-9702-09b8a78debef None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:03.845690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:03.846307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:03.846590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:03.846868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:03.847528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:03.850313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:03.857524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 459/912] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:47:03 2022] GET /identity/v3/services => generated 1268 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 12:47:03.879097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:03.883895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 454/913] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:04.001675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:04.109292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c46bd-fdc3-41fb-b774-13af3d48cd45 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:04.143604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:04.144212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:04.144515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:04.144790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:04.145438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:04.146828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:04.171439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8f79d-f4c0-4a44-8d3c-b4ce6eab7ae7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:04.214200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:04.220532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 460/914] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:47:04 2022] POST /identity/v3/services => generated 241 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:04.304443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8f79d-f4c0-4a44-8d3c-b4ce6eab7ae7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:04.350544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:04.351306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:04.351554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:04.351874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:04.363741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:04.368635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:04.369588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:04.370509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 455/915] 10.222.0.72 () {56 vars in 1120 bytes} [Tue Sep 13 12:47:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:04.378487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:04.493507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f28bd-519f-43a1-b141-bae1fe99e193 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:04.528407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:04.529019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:04.529301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:04.529574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:04.530272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:04.530577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:04.531931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:04.540706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 461/916] 10.222.0.72 () {56 vars in 1115 bytes} [Tue Sep 13 12:47:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:04.546325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:04.603182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:04.653518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae8282-2c7e-4d8e-9420-045625e4b97e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:04.688987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:04.689628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:04.689943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:04.690221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:04.701412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:04.702904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:04.717616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:04.718655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:04.719586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:04.719586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:04.719586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:04.719586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 456/917] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:04.727035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:04.791070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:04.839157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017cf7f-0c04-4dc1-a0ac-9520d8552e16 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:04.876790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:04.877869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:04.878360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:04.879000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:04.880351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:04.880887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:04.882645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:04.896352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 462/918] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:04.902233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:05.014928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca3a6-c4ef-4f0c-afe4-76092c8c9643 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:05.017420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:05.051348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:05.052106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.052349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.052536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.061237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:05.062698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:47:05.074725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:05.075977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:05.076925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 457/919] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:47:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:05.084850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:05.195004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e9a0-3eb2-4d52-b989-5fca8fe1dac7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:05.218131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:05.238317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:05.239922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.240445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.240951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.241924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:05.242434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:05.244082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:05.263835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 463/920] 10.222.0.72 () {56 vars in 1149 bytes} [Tue Sep 13 12:47:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:05.271061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:05.383347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05b9c8-2303-4842-9769-4d58363b33ed None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:05.419348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263cdf-cbda-410d-8ba5-79e8e4fe019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:05.419984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263cdf-cbda-410d-8ba5-79e8e4fe019d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.420263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263cdf-cbda-410d-8ba5-79e8e4fe019d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.420537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263cdf-cbda-410d-8ba5-79e8e4fe019d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.431323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263cdf-cbda-410d-8ba5-79e8e4fe019d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:05.432763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263cdf-cbda-410d-8ba5-79e8e4fe019d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-60263cdf-cbda-410d-8ba5-79e8e4fe019d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:05.448009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:05.448995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:05.450298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:05.450298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:05.450298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:05.450298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 458/921] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:05.456617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:05.468188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618ac9b-72a7-4c58-95ac-1c1a7d31ac94 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.468531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618ac9b-72a7-4c58-95ac-1c1a7d31ac94 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.468813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618ac9b-72a7-4c58-95ac-1c1a7d31ac94 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.470322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 459/922] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:05.476196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459fd74-b52e-46d7-b1f1-78757b49ac87 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.476476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459fd74-b52e-46d7-b1f1-78757b49ac87 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.476751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459fd74-b52e-46d7-b1f1-78757b49ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.496033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f459fd74-b52e-46d7-b1f1-78757b49ac87 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:05.497295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f459fd74-b52e-46d7-b1f1-78757b49ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:05.510109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:05.568436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 460/923] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:05.570003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955b5f9-72af-4212-a655-719d6a23327a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:05.606774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:05.607386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.607668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.607943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.608594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:05.608898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:05.610291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:05.623453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 464/924] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:05.630161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459fd74-b52e-46d7-b1f1-78757b49ac87 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:05.736398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459fd74-b52e-46d7-b1f1-78757b49ac87 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:05.776840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:05.777479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.777779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.778068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.785505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:05.788227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:47:05.800276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:05.802005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:05.804138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 461/925] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:05.807858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:05.855936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:05.865271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:05.866982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:05.941323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf9101-432d-4ba1-b7cf-6a2b3de54c07 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:05.970097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c17ed6-2d13-460d-a79b-7411a80ad79a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:05.981026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:05.981993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:05.982469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:05.982966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:05.983852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:05.984349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:05.986158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:06.001122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 465/926] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:06.008253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:06.017030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:06.017658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.017997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.018272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.018904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:06.020837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:06.032603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:06.033931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 12:47:06.034841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:06.038664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 462/927] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:47:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:06.039913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:06.116761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:06.139739 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:06.146516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b434761-3831-4e1d-a254-47ecce55cb5b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:06.154585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3fd0b-993c-42cc-9c66-60336f7908cb None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:06.192305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:06.192781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.193104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.193378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.194163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:06.194466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:06.195850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:06.198837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2058253-2618-4752-9bc1-cab7e663082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:06.200284 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2058253-2618-4752-9bc1-cab7e663082c None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.200815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2058253-2618-4752-9bc1-cab7e663082c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.201471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2058253-2618-4752-9bc1-cab7e663082c None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/e658441f5ed34cfbb926364644fbaed6/roles/84a9de490add48a7b98e6e4a899e0f30` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.204203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2058253-2618-4752-9bc1-cab7e663082c None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=e658441f5ed34cfbb926364644fbaed6, role_id=84a9de490add48a7b98e6e4a899e0f30)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:06.205912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2058253-2618-4752-9bc1-cab7e663082c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:06.208035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 463/928] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:47:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:06.209101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:06.312040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87109e43-af5e-4a31-8768-f80f68c529c6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:06.328827 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 466/929] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:47:06 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/e658441f5ed34cfbb926364644fbaed6/roles/84a9de490add48a7b98e6e4a899e0f30 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 12:47:06.330300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3add-a29d-4db8-861c-c0533226ebb2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.330663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3add-a29d-4db8-861c-c0533226ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.331012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3add-a29d-4db8-861c-c0533226ebb2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.332959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 467/930] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:06.338957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefa91d-7b12-49be-9bdc-3b57720482fe None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.339236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefa91d-7b12-49be-9bdc-3b57720482fe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.339683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefa91d-7b12-49be-9bdc-3b57720482fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.358966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:06.364818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.364818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.364818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.364818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:06.365152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dcefa91d-7b12-49be-9bdc-3b57720482fe None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:06.365232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:06.365302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dcefa91d-7b12-49be-9bdc-3b57720482fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:06.365371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:06.368282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:06.374336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 464/931] 10.222.0.72 () {56 vars in 1114 bytes} [Tue Sep 13 12:47:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:06.381272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:06.450592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 468/932] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:06.523106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d79c5-a505-4dac-bb1e-107527a9759f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:06.566330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:06.567963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.568442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.568910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.569785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:06.571336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:06.588542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b3aa-3427-4313-90b3-cb67f6760c49 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.588910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b3aa-3427-4313-90b3-cb67f6760c49 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.589254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817b3aa-3427-4313-90b3-cb67f6760c49 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.590843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 469/933] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:06.595767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f289e-8e9a-4133-a0c1-d3f8351421fd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.599086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f289e-8e9a-4133-a0c1-d3f8351421fd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.599086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f289e-8e9a-4133-a0c1-d3f8351421fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.617142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f84f289e-8e9a-4133-a0c1-d3f8351421fd None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:06.618384 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f84f289e-8e9a-4133-a0c1-d3f8351421fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:06.673712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 465/934] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:47:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:47:06.712361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 470/935] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:06.730546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:06.802419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:06.856554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fd305-4b46-47e7-b4b6-bfe1874b44b5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:06.902953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:06.903594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:06.903876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:06.904149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:06.904744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:06.906185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:06.916680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:06.917654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:06.918584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 466/936] 10.222.0.72 () {56 vars in 1132 bytes} [Tue Sep 13 12:47:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:06.925372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f289e-8e9a-4133-a0c1-d3f8351421fd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:06.945186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:07.025029 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:07.042325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f289e-8e9a-4133-a0c1-d3f8351421fd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.082006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.082729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.083070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.083414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.084113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:07.084480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.086013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.091264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 471/937] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:07.096853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:07.164746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61820556-c5e0-4bd8-b966-b0af482baf1d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.179934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:07.189025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:07.201887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19317b16-f8c4-4b40-ac3f-037b523536a8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.213160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.214475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.215208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.215664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.216587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.218850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:07.235731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:07.236829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:07.237695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:07.243108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 467/938] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:07.244433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:07.245632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.246486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.246944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.247392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.249036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:07.249541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.251196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.257804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 472/939] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:07.263433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:07.363082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59419ed-1321-43be-9aab-f2a8f7330ffa None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.386489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369f47-7b48-4b5f-8238-9f2e35c41e30 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.399906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.400502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.400846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.401234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.401906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:07.402271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.403870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.413803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 468/940] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:07.415241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:07.427863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.428704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.429164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.429611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.430730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.433076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.469873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:07.522369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f2b76-dbc0-4c6d-9299-6293a5b112ae None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.541997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:07.553631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 473/941] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:47:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:47:07.578279 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.579712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.580249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.580761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.581728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:07.582287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.584036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.590759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 469/942] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:07.596556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:07.723634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922cf11-e0bd-4bab-9c81-c04043e22588 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.760649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5b2f9-aa6e-42bd-ac1a-8812334e24c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.761264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5b2f9-aa6e-42bd-ac1a-8812334e24c4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.761547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5b2f9-aa6e-42bd-ac1a-8812334e24c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.761845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5b2f9-aa6e-42bd-ac1a-8812334e24c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.762545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5b2f9-aa6e-42bd-ac1a-8812334e24c4 None admin] RBAC: Adding query filter params (interface=public, service_id=a5c37795a8d2449cb4b1a74c63171dae, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:07.762854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5b2f9-aa6e-42bd-ac1a-8812334e24c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.764215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5b2f9-aa6e-42bd-ac1a-8812334e24c4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.769798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 474/943] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:47:07 2022] GET /identity/v3/endpoints?service_id=a5c37795a8d2449cb4b1a74c63171dae&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:07.775931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:07.881862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0c3b2-1ab0-4c54-a3f3-3146b19e0337 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:07.889668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:07.917185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19543f00-a951-4eb1-ac0f-93b479f826e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:07.918125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543f00-a951-4eb1-ac0f-93b479f826e1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:07.918620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543f00-a951-4eb1-ac0f-93b479f826e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:07.919085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543f00-a951-4eb1-ac0f-93b479f826e1 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:07.919906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19543f00-a951-4eb1-ac0f-93b479f826e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:07.921562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19543f00-a951-4eb1-ac0f-93b479f826e1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:07.931790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 470/944] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:47:07 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 12:47:08.594966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634004e-fb11-4038-9cd7-fc6b5f7f4960 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:08.595483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634004e-fb11-4038-9cd7-fc6b5f7f4960 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:08.595587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634004e-fb11-4038-9cd7-fc6b5f7f4960 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:08.597110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 475/945] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:08.603179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18420ce-1ce1-4eae-9fd6-72a080f90e78 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:08.603596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18420ce-1ce1-4eae-9fd6-72a080f90e78 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:08.603771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18420ce-1ce1-4eae-9fd6-72a080f90e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:08.622056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f18420ce-1ce1-4eae-9fd6-72a080f90e78 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:08.623246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f18420ce-1ce1-4eae-9fd6-72a080f90e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:08.703967 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 471/946] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:08.979833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634004e-fb11-4038-9cd7-fc6b5f7f4960 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.023794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31745d68-9a29-4744-b7ff-a76d6dbd7ce7 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.024138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31745d68-9a29-4744-b7ff-a76d6dbd7ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.024339 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31745d68-9a29-4744-b7ff-a76d6dbd7ce7 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.025943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 472/947] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:09.030510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52642944-c90e-4ccf-89a2-da62041046c4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.030854 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52642944-c90e-4ccf-89a2-da62041046c4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.031246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52642944-c90e-4ccf-89a2-da62041046c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.044687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:09.051133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-52642944-c90e-4ccf-89a2-da62041046c4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:09.052415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-52642944-c90e-4ccf-89a2-da62041046c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:09.088533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634004e-fb11-4038-9cd7-fc6b5f7f4960 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:09.128586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:09.130093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.130375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.130652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.134809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 473/948] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:09.143746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:09.149896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:09.150838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:09.152053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 476/949] 10.222.0.72 () {56 vars in 1120 bytes} [Tue Sep 13 12:47:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:09.157043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52642944-c90e-4ccf-89a2-da62041046c4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.190578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:09.260411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52642944-c90e-4ccf-89a2-da62041046c4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:09.295341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:09.295956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.296234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.296506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.297174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:09.297479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:09.298890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:09.308093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 474/950] 10.222.0.72 () {56 vars in 1115 bytes} [Tue Sep 13 12:47:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:09.313465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.417481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335f627-90e5-44e6-a661-bce8b48e9129 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:09.433016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.441030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:09.459730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:09.460644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.460928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.461198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.474558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:09.476020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:09.494659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:09.497078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:09.499226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:09.499226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:09.499226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:09.499226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 477/951] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:09.505786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.547568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:09.560319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc987c5c-6c5a-464a-a6e7-84a890e55f51 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:09.604550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:09.605377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.605769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.606145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.607204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:09.608731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:09.612346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING keystone.server.flask.common [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 12:47:09.622654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67477cb2-21f6-41dd-b553-d5e61c2e812a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:09.668068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:09.668676 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.668954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.669224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.669927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:09.670232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:09.672048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.sql.core [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 12:47:09.672048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 12:47:09.672048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [parameters: {'id': '429ee8c2e95d4e3d87d7fdcb0ab7f084', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 12:47:09.672048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81578) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 12:47:09.676887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 429ee8c2e95d4e3d87d7fdcb0ab7f084.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 429ee8c2e95d4e3d87d7fdcb0ab7f084. Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application conn.query(q) Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 12:47:09.687855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application result.read() Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 12:47:09.688785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application next(self.gen) Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application yield resource Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application next(self.gen) Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise self.value Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application session.commit() Sep 13 12:47:09.689819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self.session.flush() Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application compat.raise_( Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 12:47:09.693271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 12:47:09.695102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application conn.query(q) Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application result.read() Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 12:47:09.697230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [parameters: {'id': '429ee8c2e95d4e3d87d7fdcb0ab7f084', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.699592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:09.706490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 429ee8c2e95d4e3d87d7fdcb0ab7f084. Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 475/952] 10.222.0.72 () {60 vars in 1140 bytes} [Tue Sep 13 12:47:09 2022] POST /identity/v3/projects => generated 262 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:09.709630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.711240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 478/953] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:09.711240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.768272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:09.819725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd54d7-3d02-4ad3-aacd-40f500e0f0ec None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:09.856484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8388f4-f954-4a94-b5e5-8fcbc58bfea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:09.857105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8388f4-f954-4a94-b5e5-8fcbc58bfea1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.857379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8388f4-f954-4a94-b5e5-8fcbc58bfea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.857646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8388f4-f954-4a94-b5e5-8fcbc58bfea1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.858641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21563d-12dc-4fa4-b32a-402137f3301a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:09.864563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8388f4-f954-4a94-b5e5-8fcbc58bfea1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:09.866549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8388f4-f954-4a94-b5e5-8fcbc58bfea1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-dd8388f4-f954-4a94-b5e5-8fcbc58bfea1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:47:09.878053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:09.879804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:09.882053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 476/954] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:09.885272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9774c-79b2-4fec-b324-f59765d21457 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.885624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9774c-79b2-4fec-b324-f59765d21457 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.885845 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9774c-79b2-4fec-b324-f59765d21457 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.893891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 477/955] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:09.893891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9774c-79b2-4fec-b324-f59765d21457 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:09.901851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2870287-d356-404c-8abc-2302a5eab426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:09.902481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2870287-d356-404c-8abc-2302a5eab426 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.902764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2870287-d356-404c-8abc-2302a5eab426 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.903028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2870287-d356-404c-8abc-2302a5eab426 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.903128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:09.911665 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2870287-d356-404c-8abc-2302a5eab426 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:09.913129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2870287-d356-404c-8abc-2302a5eab426 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-c2870287-d356-404c-8abc-2302a5eab426 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:47:09.925905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:09.932039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 479/956] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:47:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b861e48-e23b-4369-8021-3792a0b59712 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:09.933205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b861e48-e23b-4369-8021-3792a0b59712 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:09.935052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b861e48-e23b-4369-8021-3792a0b59712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:09.950139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5b861e48-e23b-4369-8021-3792a0b59712 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:09.951328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5b861e48-e23b-4369-8021-3792a0b59712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:10.013358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9774c-79b2-4fec-b324-f59765d21457 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.027008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 480/957] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:10.038773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b861e48-e23b-4369-8021-3792a0b59712 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.047446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87672617-3fea-4cb4-a6bf-2eab486e4010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.048160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672617-3fea-4cb4-a6bf-2eab486e4010 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.048440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672617-3fea-4cb4-a6bf-2eab486e4010 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.048713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87672617-3fea-4cb4-a6bf-2eab486e4010 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.049605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87672617-3fea-4cb4-a6bf-2eab486e4010 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:10.049716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87672617-3fea-4cb4-a6bf-2eab486e4010 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.051547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87672617-3fea-4cb4-a6bf-2eab486e4010 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.071942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:10.078277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 478/958] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:47:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:10.164214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b861e48-e23b-4369-8021-3792a0b59712 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.195417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92193fd-3d92-4374-b059-a2d07ebcc912 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.195698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92193fd-3d92-4374-b059-a2d07ebcc912 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.195969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92193fd-3d92-4374-b059-a2d07ebcc912 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.197450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 479/959] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:10.203531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088f6a-4866-4da6-8e88-2bbb969d55bf None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.203944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088f6a-4866-4da6-8e88-2bbb969d55bf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.204314 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088f6a-4866-4da6-8e88-2bbb969d55bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.209093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.210015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.210446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.210894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.211816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:10.212274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.213913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.225866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-79088f6a-4866-4da6-8e88-2bbb969d55bf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:10.227420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-79088f6a-4866-4da6-8e88-2bbb969d55bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:10.235417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 481/960] 10.222.0.72 () {56 vars in 1149 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:10.243096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.308993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:10.316084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 480/961] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:10.319874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088f6a-4866-4da6-8e88-2bbb969d55bf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.347858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:10.371760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c1937-2d16-4ec1-856f-340753a0aaeb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.417122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.417732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.418039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.418312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.443138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088f6a-4866-4da6-8e88-2bbb969d55bf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.481757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.482400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.482677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.482947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.483529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.484971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:10.494972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:10.496180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:10.502333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 481/962] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:10.505193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.521923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.524254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.574347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:10.617826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e47b-7e0c-4e2e-91ee-ec923e334a5d None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.626739 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:10.652761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 482/963] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 12:47:10.653566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.659105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.659707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.660017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.660293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.660922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:10.661225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.662680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.667874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 482/964] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:10.668745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.768297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebdf5f-3d9b-456b-9f40-7204aca860ff None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.776592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1444681-4ce4-4f39-9a59-3b88689de341 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.803287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:10.808195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.808832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.809112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.809394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.810058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.811421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.812685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.813554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.813868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.814146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:10.820498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:10.821482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:10.822398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:10.823612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 483/965] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:10.824477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.826922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.828665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:10.844985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:10.845993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:10.846897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:10.846897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:10.846897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:10.846897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 483/966] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:10.846897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:10.881630 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:10.937767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b1ad8-e0c3-462e-a6bf-6fedbd068615 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.952757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3ca6c-d14e-4bfd-95d0-a4da0879c202 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:10.976837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:10.977449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:10.977730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:10.978060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:10.978691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:10.978992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:10.980348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:10.984969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 484/967] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:10.985762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:11.005677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:11.006345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:11.006609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:11.006881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:11.007527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:11.007830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:11.009238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:11.017798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 484/968] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:11.020424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:11.074620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:11.130002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f2cc7-6f15-4ae9-a9dc-4d7b5f27f112 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:11.145295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:11.171030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364639-74b2-48e2-bb54-3a3b4b8f2923 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:11.176045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:11.176914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:11.177332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:11.177775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:11.178579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:11.179020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:11.180535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:11.194168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 485/969] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:11.199515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:11.215623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:11.216249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:11.216529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:11.216928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:11.217574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:11.217920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:11.219520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:11.229043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 485/970] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:11.235034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:11.328896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54477d-5a4a-4189-bcb8-6f25e983f5c1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:11.365293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:11.366061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:11.366404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:11.366741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:11.373821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:11.374349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71c82c-8c58-42dd-9183-1d3f463d61ae None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:11.375465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:11.382792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:47:11.391538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:11.392442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:11.393347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:11.400610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 486/971] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:11.401277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:11.424312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:11.425928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:11.426690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:11.427174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:11.428296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:11.430535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:11.434425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:11.511406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1df201-5573-4071-87e2-f3ab0d122c30 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:11.514450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 486/972] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:47:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:47:11.564171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93212-d96b-4c9c-a109-04da617575f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:11.564877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93212-d96b-4c9c-a109-04da617575f7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:11.565161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93212-d96b-4c9c-a109-04da617575f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:11.565430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93212-d96b-4c9c-a109-04da617575f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:11.566290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93212-d96b-4c9c-a109-04da617575f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:11.566592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93212-d96b-4c9c-a109-04da617575f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:11.568204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93212-d96b-4c9c-a109-04da617575f7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:11.583239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 487/973] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:11.589350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:11.706255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2e426-fd01-45f7-a630-e6cf7389f636 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:11.744324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e28f9-654c-4e84-a4b8-d9efff605078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:11.744935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e28f9-654c-4e84-a4b8-d9efff605078 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:11.745215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e28f9-654c-4e84-a4b8-d9efff605078 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:11.745489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e28f9-654c-4e84-a4b8-d9efff605078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:11.746256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e28f9-654c-4e84-a4b8-d9efff605078 None admin] RBAC: Adding query filter params (role.id=84a9de490add48a7b98e6e4a899e0f30, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=e658441f5ed34cfbb926364644fbaed6) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:11.746563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e28f9-654c-4e84-a4b8-d9efff605078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:11.748004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e28f9-654c-4e84-a4b8-d9efff605078 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:11.754852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 487/974] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:11 2022] GET /identity/v3/role_assignments?user.id=e658441f5ed34cfbb926364644fbaed6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=84a9de490add48a7b98e6e4a899e0f30 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:12.532112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762292b-bd5b-4413-a9a3-d7a29a286519 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:12.532650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762292b-bd5b-4413-a9a3-d7a29a286519 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:12.532650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762292b-bd5b-4413-a9a3-d7a29a286519 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:12.534320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 488/975] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:12.540288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ab2-ff77-45e3-89a8-21040c6fde09 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:12.540575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ab2-ff77-45e3-89a8-21040c6fde09 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:12.540848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ab2-ff77-45e3-89a8-21040c6fde09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:12.560439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-60707ab2-ff77-45e3-89a8-21040c6fde09 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:12.561619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-60707ab2-ff77-45e3-89a8-21040c6fde09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:12.582971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:12.635584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 488/976] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:12.905619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762292b-bd5b-4413-a9a3-d7a29a286519 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:12.996190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:13.013137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762292b-bd5b-4413-a9a3-d7a29a286519 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:13.048660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:13.049318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.049596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.049890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.050590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:13.052008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:13.054621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING keystone.server.flask.common [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 12:47:13.093571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.sql.core [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 12:47:13.093571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 12:47:13.093571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [parameters: {'user_id': '1a2b9e5cef0e4af9b41cc1fc4cbb9e56', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 12:47:13.093571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 13 12:47:13.100443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 12:47:13.101331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application yield resource Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application next(self.gen) Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise self.value Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application session.commit() Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 12:47:13.102319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.session.flush() Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application compat.raise_( Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 12:47:13.103217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 12:47:13.104104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application conn.query(q) Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application result.read() Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 12:47:13.104958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [parameters: {'user_id': '1a2b9e5cef0e4af9b41cc1fc4cbb9e56', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:13.106976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 12:47:13.108882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 12:47:13.111055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 12:47:13.111055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 12:47:13.111055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 12:47:13.111055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 12:47:13.111055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 12:47:13.111055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:13.111055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 489/977] 10.222.0.72 () {60 vars in 1131 bytes} [Tue Sep 13 12:47:12 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:13.111953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707ab2-ff77-45e3-89a8-21040c6fde09 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:13.217851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707ab2-ff77-45e3-89a8-21040c6fde09 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:13.254452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6969c2-d027-4f65-89ce-ddd7cbfbb1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:13.255095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6969c2-d027-4f65-89ce-ddd7cbfbb1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.255372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6969c2-d027-4f65-89ce-ddd7cbfbb1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.255639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6969c2-d027-4f65-89ce-ddd7cbfbb1c9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.263253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6969c2-d027-4f65-89ce-ddd7cbfbb1c9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:13.264739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6969c2-d027-4f65-89ce-ddd7cbfbb1c9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2b6969c2-d027-4f65-89ce-ddd7cbfbb1c9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:47:13.277013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:13.279167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 12:47:13.281354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:13.286771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 489/978] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:47:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:13.286925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:13.329770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738968c4-8969-44a2-9874-95ef0f441006 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.329770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738968c4-8969-44a2-9874-95ef0f441006 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.329770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738968c4-8969-44a2-9874-95ef0f441006 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.330854 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 490/979] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:13.336699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db75ef-5ba6-4fb2-b563-cf54d34043c9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.337220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db75ef-5ba6-4fb2-b563-cf54d34043c9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.337696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db75ef-5ba6-4fb2-b563-cf54d34043c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.356747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-03db75ef-5ba6-4fb2-b563-cf54d34043c9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:13.358294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-03db75ef-5ba6-4fb2-b563-cf54d34043c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:13.373511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:13.401399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436fc5b-a173-47e1-a961-f10fa01fd9e8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:13.450029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9aa5-8cd1-4fda-8f91-4cafe415d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:13.450029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9aa5-8cd1-4fda-8f91-4cafe415d82d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.450029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9aa5-8cd1-4fda-8f91-4cafe415d82d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.450029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9aa5-8cd1-4fda-8f91-4cafe415d82d None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.450452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9aa5-8cd1-4fda-8f91-4cafe415d82d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:13.450452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9aa5-8cd1-4fda-8f91-4cafe415d82d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:13.450897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 491/980] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:13.451286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9aa5-8cd1-4fda-8f91-4cafe415d82d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:13.497999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 490/981] 10.222.0.72 () {56 vars in 1101 bytes} [Tue Sep 13 12:47:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:13.734629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db75ef-5ba6-4fb2-b563-cf54d34043c9 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:13.814152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb1e48-a4fc-4103-95c5-5ff1d468e01f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.814638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb1e48-a4fc-4103-95c5-5ff1d468e01f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.814638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb1e48-a4fc-4103-95c5-5ff1d468e01f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.816635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 491/982] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:13.821454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822324-81a8-4de5-a392-5547c1d45f74 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.821760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822324-81a8-4de5-a392-5547c1d45f74 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.822051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822324-81a8-4de5-a392-5547c1d45f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.835444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:13.841202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5a822324-81a8-4de5-a392-5547c1d45f74 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:13.842756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a822324-81a8-4de5-a392-5547c1d45f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:13.844457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db75ef-5ba6-4fb2-b563-cf54d34043c9 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:13.885098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e3c33-b258-4d07-ba74-c786777fe420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:13.885726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3c33-b258-4d07-ba74-c786777fe420 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.886035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3c33-b258-4d07-ba74-c786777fe420 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.886306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3c33-b258-4d07-ba74-c786777fe420 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.887145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e3c33-b258-4d07-ba74-c786777fe420 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:13.888673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e3c33-b258-4d07-ba74-c786777fe420 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:13.917074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 492/983] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:13.981139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 492/984] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:47:13 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:13.997371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96324d-5389-4926-b0e6-57962248973a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:13.997689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96324d-5389-4926-b0e6-57962248973a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:13.997977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96324d-5389-4926-b0e6-57962248973a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:13.999575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 493/985] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:14.006262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c8845-d21c-4bf9-bd0f-fae4a7de7072 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.006499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c8845-d21c-4bf9-bd0f-fae4a7de7072 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.006780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c8845-d21c-4bf9-bd0f-fae4a7de7072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.020653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:14.027218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-187c8845-d21c-4bf9-bd0f-fae4a7de7072 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:14.028574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-187c8845-d21c-4bf9-bd0f-fae4a7de7072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:14.111674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 493/986] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:14.188776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96324d-5389-4926-b0e6-57962248973a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.293982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96324d-5389-4926-b0e6-57962248973a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.328703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.329379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.329663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.329960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.330536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.332257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:14.334845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:14.341952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:14.342957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:14.343875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:14.345810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 494/987] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:14.356386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c8845-d21c-4bf9-bd0f-fae4a7de7072 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.378038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.468936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c8845-d21c-4bf9-bd0f-fae4a7de7072 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.483797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451a767-3be5-4942-874c-b11a5212de92 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.501416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:14.507805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.508422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.508807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.509078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.509711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:14.510035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.511404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:14.517778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 494/988] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:14.521686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.524651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.525284 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.526330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.526330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] PATH_INFO: `/v3/services/image` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.526580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.527932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:14.537659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:14.538879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:14.539910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:14.543615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 495/989] 10.222.0.72 () {56 vars in 1105 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:14.544445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.611679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:14.625171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaede7-9d0f-4a02-952d-9d9621dc0f8f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.658449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030f288-bef3-458d-97e7-adeebaf30efe None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.666597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.667195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.667477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.667746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.668418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:14.668719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.670099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:14.674838 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 495/990] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:14.681149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.702069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.702718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.702995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.703266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.703891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] RBAC: Adding query filter params (name=image) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:14.704190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.706055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:14.711177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 496/991] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:14.716501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.740698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:14.781952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada50d-eea4-41e8-8102-c8ba61f7fb65 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.816910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.817562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.817887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.818165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.818869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] RBAC: Adding query filter params (interface=admin, service_id=a5c37795a8d2449cb4b1a74c63171dae, region_id=RegionOne) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:14.819178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.820530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:14.823913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353e497-52ff-4034-b370-7e12f866ee93 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.826445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 496/992] 10.222.0.72 () {56 vars in 1249 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/endpoints?service_id=a5c37795a8d2449cb4b1a74c63171dae&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:14.832382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:14.853039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:14.861802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da53fd0-3f1f-4184-b453-12b88b97061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.862409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da53fd0-3f1f-4184-b453-12b88b97061b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.863969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da53fd0-3f1f-4184-b453-12b88b97061b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.863969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da53fd0-3f1f-4184-b453-12b88b97061b None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.863969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da53fd0-3f1f-4184-b453-12b88b97061b None admin] RBAC: Adding query filter params (type=image) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:14.863969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da53fd0-3f1f-4184-b453-12b88b97061b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.865443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da53fd0-3f1f-4184-b453-12b88b97061b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:14.881260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 497/993] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:14.951162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77eae4-9f5b-4288-a0cc-b0f27bfed418 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:14.977961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:14.989303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb9ed5-cb17-41c0-8fd0-588d36e70d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:14.990160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb9ed5-cb17-41c0-8fd0-588d36e70d5a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:14.990275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb9ed5-cb17-41c0-8fd0-588d36e70d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:14.990498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb9ed5-cb17-41c0-8fd0-588d36e70d5a None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:14.991122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb9ed5-cb17-41c0-8fd0-588d36e70d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:14.992502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb9ed5-cb17-41c0-8fd0-588d36e70d5a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:15.006259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 497/994] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:47:14 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 12:47:16.279820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d556dc7-178f-4c2c-b659-11868db73d20 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:16.280342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d556dc7-178f-4c2c-b659-11868db73d20 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:16.280420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d556dc7-178f-4c2c-b659-11868db73d20 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:16.281891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 498/995] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:16.287541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80778096-cdc1-49a8-9e11-016ac8685841 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:16.287944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80778096-cdc1-49a8-9e11-016ac8685841 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:16.288114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80778096-cdc1-49a8-9e11-016ac8685841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:16.306576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-80778096-cdc1-49a8-9e11-016ac8685841 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:16.307762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-80778096-cdc1-49a8-9e11-016ac8685841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:16.380698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 498/996] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:16.658860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d556dc7-178f-4c2c-b659-11868db73d20 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:16.763840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d556dc7-178f-4c2c-b659-11868db73d20 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:16.798278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:16.798914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:16.799200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:16.799479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:16.800072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:16.802264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:16.811281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:16.812268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:16.813126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:16.818051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 499/997] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:16.819598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80778096-cdc1-49a8-9e11-016ac8685841 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:16.922996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80778096-cdc1-49a8-9e11-016ac8685841 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:16.940286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:16.960029 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e224d-db04-46ee-b527-0bb803931b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:16.960660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e224d-db04-46ee-b527-0bb803931b5a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:16.960951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e224d-db04-46ee-b527-0bb803931b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:16.961202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e224d-db04-46ee-b527-0bb803931b5a None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:16.961881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e224d-db04-46ee-b527-0bb803931b5a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:16.962182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e224d-db04-46ee-b527-0bb803931b5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:16.963590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e224d-db04-46ee-b527-0bb803931b5a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:16.973968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 499/998] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:47:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:16.975694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:17.031683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:17.075532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b378a3f-0417-4571-8cd1-9bdfb737d920 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.075817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b378a3f-0417-4571-8cd1-9bdfb737d920 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.076091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b378a3f-0417-4571-8cd1-9bdfb737d920 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.077724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 500/999] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:17.082244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7a649-b3ce-4283-95df-0e3fb2dce5f2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:17.083314 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208f982-c934-4cff-a580-fe9ab564dec8 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.083596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208f982-c934-4cff-a580-fe9ab564dec8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.083874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208f982-c934-4cff-a580-fe9ab564dec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.102915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e208f982-c934-4cff-a580-fe9ab564dec8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:17.104184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e208f982-c934-4cff-a580-fe9ab564dec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:17.117984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:17.118588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.118865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.119135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.119761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:17.120060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:17.121406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:17.131184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 500/1000] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:47:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:17.132318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:17.181774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 501/1001] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:17.182807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69802d80-9baa-4bd0-83a0-4dc8703e8c4e None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.183090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69802d80-9baa-4bd0-83a0-4dc8703e8c4e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.183357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69802d80-9baa-4bd0-83a0-4dc8703e8c4e None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.184698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 502/1002] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:17.191183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc410-5e0d-4203-80a3-60ced9b2ea5a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.191508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc410-5e0d-4203-80a3-60ced9b2ea5a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.191776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfc410-5e0d-4203-80a3-60ced9b2ea5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.213601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-48cfc410-5e0d-4203-80a3-60ced9b2ea5a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:17.215174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-48cfc410-5e0d-4203-80a3-60ced9b2ea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:17.241360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e59bfe-033a-4bab-a05f-ca99fffba81a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:17.262714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:17.277358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:17.277994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.278282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.278699 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.279349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] RBAC: Adding query filter params (interface=public, service_id=0c3e5f3f08cd42fa9ee26f844842a792, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:17.279593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:17.280945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:17.287369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 501/1003] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:47:17 2022] GET /identity/v3/endpoints?service_id=0c3e5f3f08cd42fa9ee26f844842a792&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:17.297807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 503/1004] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:17.298246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:17.407963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d0dd-5bb2-4401-9a70-06592c152cc7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:17.449120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:17.450006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.450277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.450548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.451146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:17.452574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:17.452987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfc410-5e0d-4203-80a3-60ced9b2ea5a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:17.465808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 502/1005] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:47:17 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 12:47:17.504678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:17.568387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfc410-5e0d-4203-80a3-60ced9b2ea5a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:17.586729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:17.608602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:17.609128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.609405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.609677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.610389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:17.611792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:17.645837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:17.690025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 504/1006] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:47:17 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:47:17.700902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a381c3-5a15-4f9f-a7ef-cb7e9460fe16 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:17.747159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:17.747819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.748177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.748476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.749061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:17.750512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:17.760980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:17.762347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:17.763629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 503/1007] 10.222.0.72 () {56 vars in 1126 bytes} [Tue Sep 13 12:47:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:17.768234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:17.885907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75aacf-71d0-400e-8fef-e7827a271c72 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:17.887555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:17.921463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:17.922812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:17.922812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:17.922812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:17.923368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:17.923674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:17.925042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:17.930478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 505/1008] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:17.936655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:18.042749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4450-1f7c-4c72-b1ac-444a684c9893 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:18.052220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:18.077621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc909a1-5527-417d-83ae-6f0e7981f86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:18.078335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc909a1-5527-417d-83ae-6f0e7981f86c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:18.078680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc909a1-5527-417d-83ae-6f0e7981f86c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:18.079021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc909a1-5527-417d-83ae-6f0e7981f86c None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:18.079724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc909a1-5527-417d-83ae-6f0e7981f86c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:18.080088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc909a1-5527-417d-83ae-6f0e7981f86c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:18.081506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc909a1-5527-417d-83ae-6f0e7981f86c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:18.090893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 504/1009] 10.222.0.72 () {56 vars in 1122 bytes} [Tue Sep 13 12:47:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:18.092118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:18.195575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192f01f-cd7e-42a5-b974-7e6407ac8339 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:18.231203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcffd7d-5e10-48d1-80bc-31bba0432c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:18.231808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcffd7d-5e10-48d1-80bc-31bba0432c9a None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:18.232086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcffd7d-5e10-48d1-80bc-31bba0432c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:18.232357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcffd7d-5e10-48d1-80bc-31bba0432c9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:18.233022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcffd7d-5e10-48d1-80bc-31bba0432c9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:18.234415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcffd7d-5e10-48d1-80bc-31bba0432c9a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:18.330262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 506/1010] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:47:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:19.678383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccea5e9-10b3-46b0-9b0f-ca844c27bc93 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:19.678935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccea5e9-10b3-46b0-9b0f-ca844c27bc93 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:19.678935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccea5e9-10b3-46b0-9b0f-ca844c27bc93 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:19.680439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 505/1011] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:19.685421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b860-4ea3-42fc-bbef-4f07121927ca None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:19.685905 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b860-4ea3-42fc-bbef-4f07121927ca None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:19.686005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b860-4ea3-42fc-bbef-4f07121927ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:19.704706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-de21b860-4ea3-42fc-bbef-4f07121927ca None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:19.705930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-de21b860-4ea3-42fc-bbef-4f07121927ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:19.727913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:19.783484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 507/1012] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:19.887294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9fc26-c8a3-40ad-a07e-3cbf80147869 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:19.887483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9fc26-c8a3-40ad-a07e-3cbf80147869 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:19.887873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9fc26-c8a3-40ad-a07e-3cbf80147869 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:19.889314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 506/1013] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:19.895211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e9d1-39c7-49fa-97dd-e8f17c4fc2d4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:19.895491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e9d1-39c7-49fa-97dd-e8f17c4fc2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:19.895759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e9d1-39c7-49fa-97dd-e8f17c4fc2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:19.914232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f2a9e9d1-39c7-49fa-97dd-e8f17c4fc2d4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:19.915415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f2a9e9d1-39c7-49fa-97dd-e8f17c4fc2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:19.984468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 508/1014] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:20.056527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9fc26-c8a3-40ad-a07e-3cbf80147869 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.160266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9fc26-c8a3-40ad-a07e-3cbf80147869 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:20.194453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:20.195079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.195363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.195639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.196224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:20.197566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:20.206564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:20.207524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 12:47:20.208357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:20.211158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 507/1015] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:20.213802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9e9d1-39c7-49fa-97dd-e8f17c4fc2d4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.252804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.296793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:20.297558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:20.315425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9e9d1-39c7-49fa-97dd-e8f17c4fc2d4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:20.357322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:20.357958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.358250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.358521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.359154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:20.359457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:20.360816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:20.365448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd027d-9d0f-4d23-8463-b21508a78283 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:20.369790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 509/1016] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:20.372252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.406424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:20.407083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.407352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.407624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] PATH_INFO: `/v3/services/image` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.408226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:20.409677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:20.419531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:20.420585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:20.422099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 508/1017] 10.222.0.72 () {56 vars in 1105 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:20.427478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.492810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc96b-5538-48f0-ad68-1dd2d7489ad4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:20.530390 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197271b-09c0-4d67-96b6-a5281ecf323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:20.530596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:20.530998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197271b-09c0-4d67-96b6-a5281ecf323b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.531460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197271b-09c0-4d67-96b6-a5281ecf323b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.531634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197271b-09c0-4d67-96b6-a5281ecf323b None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.532249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197271b-09c0-4d67-96b6-a5281ecf323b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:20.532543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197271b-09c0-4d67-96b6-a5281ecf323b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:20.533652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb500816-829f-4431-a701-ce590e0f327f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:20.533939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197271b-09c0-4d67-96b6-a5281ecf323b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:20.541708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 510/1018] 10.222.0.72 () {56 vars in 1112 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:20.541708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef0ff5-ec2b-482b-89f4-edbe139621d1 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.541708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef0ff5-ec2b-482b-89f4-edbe139621d1 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.541708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef0ff5-ec2b-482b-89f4-edbe139621d1 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.545966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 511/1019] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:20.545966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef0ff5-ec2b-482b-89f4-edbe139621d1 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.557092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:20.570087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8c5b4-cde1-477c-94b9-67f7567c15af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:20.570696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8c5b4-cde1-477c-94b9-67f7567c15af None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.575671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8c5b4-cde1-477c-94b9-67f7567c15af None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.575671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8c5b4-cde1-477c-94b9-67f7567c15af None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.575671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8c5b4-cde1-477c-94b9-67f7567c15af None admin] RBAC: Adding query filter params (name=image) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:20.575671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8c5b4-cde1-477c-94b9-67f7567c15af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:20.575671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8c5b4-cde1-477c-94b9-67f7567c15af None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:20.578564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 509/1020] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:20.579769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856a674-d757-41b5-bf0b-742b7afd41d8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.580064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856a674-d757-41b5-bf0b-742b7afd41d8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.580341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856a674-d757-41b5-bf0b-742b7afd41d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.599181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9856a674-d757-41b5-bf0b-742b7afd41d8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:20.600484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9856a674-d757-41b5-bf0b-742b7afd41d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:20.656836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef0ff5-ec2b-482b-89f4-edbe139621d1 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:20.675284 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 510/1021] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:20.676840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856a674-d757-41b5-bf0b-742b7afd41d8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.697950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:20.698554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.698833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.699104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.700018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:20.701381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:20.768837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:20.775557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856a674-d757-41b5-bf0b-742b7afd41d8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:20.786517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 512/1022] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:47:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:47:20.819859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:20.820488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:20.820768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:20.821038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:20.821683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] RBAC: Adding query filter params (type=image) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:20.822012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:20.823408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:20.834820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 511/1023] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:20.838878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.917038 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:20.963456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:20.965072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1b5d3-3978-4437-8e48-c8bfe0907298 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:21.005019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:21.006193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:21.006193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:21.006368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:21.007701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] RBAC: Adding query filter params (interface=public, service_id=6d567d442c2f4096b435a14c6837439d, region_id=RegionOne) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:21.007701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:21.009392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:21.014066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 513/1024] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/endpoints?service_id=6d567d442c2f4096b435a14c6837439d&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:21.019908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:21.085622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb2618-07c3-4d67-b6df-c13c48bbf283 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:21.121701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:21.122361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:21.122642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:21.122923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:21.124025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:21.125443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:21.132141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 512/1025] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:21.132663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0140-cd17-419d-bbe6-f285f6571588 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:21.133421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:21.157940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:21.169104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:21.176052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da744502-5c9e-493d-b6d3-2ef94875f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:21.177022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da744502-5c9e-493d-b6d3-2ef94875f9fa None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:21.177341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da744502-5c9e-493d-b6d3-2ef94875f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:21.177770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da744502-5c9e-493d-b6d3-2ef94875f9fa None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:21.178715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da744502-5c9e-493d-b6d3-2ef94875f9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:21.180256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da744502-5c9e-493d-b6d3-2ef94875f9fa None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:21.193768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 514/1026] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:47:21 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 12:47:21.249595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98b631-513c-4044-abc8-98a367ab078c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:21.288411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ced61-b435-4bc8-8eab-09f558f69745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:21.289030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ced61-b435-4bc8-8eab-09f558f69745 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:21.289312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ced61-b435-4bc8-8eab-09f558f69745 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:21.289588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ced61-b435-4bc8-8eab-09f558f69745 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:21.290299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ced61-b435-4bc8-8eab-09f558f69745 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:21.291745 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ced61-b435-4bc8-8eab-09f558f69745 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:21.416612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 513/1027] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:47:21 2022] POST /identity/v3/projects => generated 311 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:47:23.093545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a0718-cc3b-43a8-a717-c53066cdc12f None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.094134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a0718-cc3b-43a8-a717-c53066cdc12f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.094134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a0718-cc3b-43a8-a717-c53066cdc12f None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.095641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 515/1028] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:23.100771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c4af6-1343-4bc7-a27d-df3feef137f5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.101209 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c4af6-1343-4bc7-a27d-df3feef137f5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.101383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c4af6-1343-4bc7-a27d-df3feef137f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.120082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e32c4af6-1343-4bc7-a27d-df3feef137f5 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:23.121270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e32c4af6-1343-4bc7-a27d-df3feef137f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:23.188220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 514/1029] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:23.379843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2fe91-b750-4ec8-a20d-3d08a57230a4 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.380135 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2fe91-b750-4ec8-a20d-3d08a57230a4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.380302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2fe91-b750-4ec8-a20d-3d08a57230a4 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.381811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 516/1030] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:23.386866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a8a0e-bce6-4c43-b931-2d4ce5fb46f0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.387156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a8a0e-bce6-4c43-b931-2d4ce5fb46f0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.387431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a8a0e-bce6-4c43-b931-2d4ce5fb46f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.394698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:23.405933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9a5a8a0e-bce6-4c43-b931-2d4ce5fb46f0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:23.409823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9a5a8a0e-bce6-4c43-b931-2d4ce5fb46f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:23.471506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2fe91-b750-4ec8-a20d-3d08a57230a4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:23.483840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 515/1031] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:23.581794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2fe91-b750-4ec8-a20d-3d08a57230a4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:23.615100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:23.615727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.616001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.616269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.626692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:23.637502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:23.638731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:23.639789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 517/1032] 10.222.0.72 () {56 vars in 1120 bytes} [Tue Sep 13 12:47:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:23.644711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:23.644872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5ab1c-dddd-48d4-bfe7-7306d704da78 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.644872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5ab1c-dddd-48d4-bfe7-7306d704da78 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.644872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5ab1c-dddd-48d4-bfe7-7306d704da78 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.646435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 516/1033] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:23.651586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06df9e9-32a6-402b-a7df-2994d3c2ad81 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.651872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06df9e9-32a6-402b-a7df-2994d3c2ad81 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.652147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06df9e9-32a6-402b-a7df-2994d3c2ad81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.678789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a06df9e9-32a6-402b-a7df-2994d3c2ad81 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:23.680269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a06df9e9-32a6-402b-a7df-2994d3c2ad81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:23.702993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:23.748902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b6e2e-a0e7-46ca-8115-9ecedb30f041 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:23.771480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 517/1034] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:23.772334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06df9e9-32a6-402b-a7df-2994d3c2ad81 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:23.785637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:23.786271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.786549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.786827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.787480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:23.787781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:23.789135 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:23.797354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 518/1035] 10.222.0.72 () {56 vars in 1115 bytes} [Tue Sep 13 12:47:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:23.804010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:23.839465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:23.878693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06df9e9-32a6-402b-a7df-2994d3c2ad81 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:23.912596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a7f57-d232-422d-8413-73130795616f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:23.915072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:23.915766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.920280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.920280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] PATH_INFO: `/v3/services/image` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.920280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:23.920280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:23.928329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:23.929333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:23.930224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 518/1036] 10.222.0.72 () {56 vars in 1105 bytes} [Tue Sep 13 12:47:23 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:23.934867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:23.935380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:23.954994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:23.955622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:23.955897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:23.956166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:23.968174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:23.971995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:23.984605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:23.985608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:23.986560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:23.986560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:23.986560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:23.986560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 519/1037] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:23.991472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.045989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352c7a3-eeb6-4ec8-b130-4a3b72537799 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.073482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:24.081915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.083090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.083425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.083663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.084295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] RBAC: Adding query filter params (name=image) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:24.084597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.086007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.095615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 519/1038] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:24.095615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.095831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbc88d-d51b-45e5-8e91-d0b8bc9ada60 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.128384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.129191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.133651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.133651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.133651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:24.133651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.133651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.143662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 520/1039] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:24.145548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.173181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:24.193043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c63c25-3733-476d-99ca-04bb77331d00 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.227228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.227861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.228144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.228421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.229545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.231436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.233857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 520/1040] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:24.234638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.246593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75579b-47d8-4567-bc3a-51504d2cb8ec None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.280608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.281209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.281486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.281779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.282410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] RBAC: Adding query filter params (type=image) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:24.282709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.284056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.288817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 521/1041] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:24.289639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.293937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:24.335768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a2c2b-fe74-459c-9fcf-1bdcb086d14c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.370245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.370867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.371151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.371428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.379496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.380915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.381562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:24.389761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e881b5b-6e5e-40ab-b9fe-911fbc63e793 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:47:24.398425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:24.399537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 521/1042] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:47:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:24.400704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.424049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.424577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.424849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.425167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.426008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.427413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.496948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005e248-0cf9-4971-9743-f2afa3b109fb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.510659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 522/1043] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:47:24 2022] POST /identity/v3/users => generated 285 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:24.511562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.517704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:24.535952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.536582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.536868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.537143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.538526 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.539452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.623681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c633e-556d-4888-8c0c-f263beefc4f5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.648537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 522/1044] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:47:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:24.688686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.689242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.689516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.689822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.690579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:24.690893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.692338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:24.714765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 523/1045] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:47:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:24.721230 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:24.839062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d362c-556e-4b6c-b491-d7fcb1ac6e68 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:24.874950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:24.875558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:24.875832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:24.876142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:24.922464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:24.945842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:24.947214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:25.047045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 523/1046] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:47:25.053789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:25.056893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:25.158471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624798b-259c-4611-b36b-dd8ba285914a None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:25.193848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:25.194480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:25.194757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:25.195028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:25.205225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:25.207356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:25.221773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:25.222783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:25.228423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:25.228423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:25.228423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:25.228423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 524/1047] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:25.230098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:25.307412 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:25.333475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8a463-b1ab-4f82-ace1-770011f9aadc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:25.367846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:25.368535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:25.368823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:25.369099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:25.369765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:25.370070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:25.371434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:25.388704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 524/1048] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:25.390534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:25.446782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:25.496581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc06ca-b247-464b-b9a0-065d1ca5fdf4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:25.530106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:25.530730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:25.531008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:25.531277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:25.538754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:25.540183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:47:25.551758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:25.552642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:25.558716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 525/1049] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:25.560893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:25.667120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e1aa-b991-4779-b93f-5b3f2168301e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:25.701379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af21e40-23ca-4aa0-a9c8-6d9edab321c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:25.702000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af21e40-23ca-4aa0-a9c8-6d9edab321c4 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:25.702278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af21e40-23ca-4aa0-a9c8-6d9edab321c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:25.702548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af21e40-23ca-4aa0-a9c8-6d9edab321c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:25.703227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af21e40-23ca-4aa0-a9c8-6d9edab321c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:25.703528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af21e40-23ca-4aa0-a9c8-6d9edab321c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:25.704935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af21e40-23ca-4aa0-a9c8-6d9edab321c4 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:25.708317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:25.719021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 525/1050] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:25.724326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:25.828509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdf42c-74e9-4209-b103-3dc473c3a321 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:25.842802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:25.864077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc87228-2de1-4531-b21d-ef161e77cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:25.864685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87228-2de1-4531-b21d-ef161e77cb7e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:25.864963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87228-2de1-4531-b21d-ef161e77cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:25.865235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc87228-2de1-4531-b21d-ef161e77cb7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:25.865989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87228-2de1-4531-b21d-ef161e77cb7e None admin] RBAC: Adding query filter params (role.id=84a9de490add48a7b98e6e4a899e0f30, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=3b21d991d02b4f95a8d0fcec0a334495) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:25.866302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87228-2de1-4531-b21d-ef161e77cb7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:25.867724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc87228-2de1-4531-b21d-ef161e77cb7e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:25.882045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 526/1051] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:25 2022] GET /identity/v3/role_assignments?user.id=3b21d991d02b4f95a8d0fcec0a334495&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=84a9de490add48a7b98e6e4a899e0f30 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:26.770736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559369fc-9df5-4d19-9c1d-b36390c6aeae None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:26.771255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559369fc-9df5-4d19-9c1d-b36390c6aeae None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:26.771255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559369fc-9df5-4d19-9c1d-b36390c6aeae None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:26.772652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 526/1052] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:26.778033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7daba9-4ab3-4330-aea2-313ec8ad6691 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:26.778471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7daba9-4ab3-4330-aea2-313ec8ad6691 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:26.778616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7daba9-4ab3-4330-aea2-313ec8ad6691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:26.797364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-fa7daba9-4ab3-4330-aea2-313ec8ad6691 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:26.798564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fa7daba9-4ab3-4330-aea2-313ec8ad6691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:26.861126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931bd7-50c7-4eca-9edd-cc9856de4799 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:26.861417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931bd7-50c7-4eca-9edd-cc9856de4799 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:26.861673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931bd7-50c7-4eca-9edd-cc9856de4799 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:26.863733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 527/1053] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:26.868636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99844018-7560-4299-b453-c08ea3509925 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:26.868921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99844018-7560-4299-b453-c08ea3509925 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:26.872826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99844018-7560-4299-b453-c08ea3509925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:26.872937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 527/1054] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:26.893718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-99844018-7560-4299-b453-c08ea3509925 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:26.897050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-99844018-7560-4299-b453-c08ea3509925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:26.981584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 528/1055] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:27.146291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7daba9-4ab3-4330-aea2-313ec8ad6691 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:27.244334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7daba9-4ab3-4330-aea2-313ec8ad6691 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:27.253675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:27.255131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99844018-7560-4299-b453-c08ea3509925 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:27.281269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:27.281929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:27.282204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:27.282473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:27.292574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:27.297666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:27.298834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:27.299713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 528/1056] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:27.305319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:27.361694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99844018-7560-4299-b453-c08ea3509925 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:27.389374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:27.399614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:27.400150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:27.400429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:27.400701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:27.408212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9446c3-158e-4f7e-a431-a77e9e5b7004 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:27.442312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:27.442919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:27.443197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:27.443477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:27.444141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:27.444445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:27.445833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:27.454657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 529/1057] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:27.463348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:27.477398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:27.478887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:27.494071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:27.568483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d13599-0bd6-417e-b9d7-1fe12c25e9fd None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:27.587112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 529/1058] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 12:47:27.592557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:27.605918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:27.606551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:27.606828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:27.607096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] PATH_INFO: `/v3/users/4f89684343e0436b9b66736f008c5c30` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:27.620930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] RBAC: Authorizing `identity:get_user(user_id=4f89684343e0436b9b66736f008c5c30)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:27.621255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:27.622393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:27.624708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 530/1059] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/users/4f89684343e0436b9b66736f008c5c30 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:27.634179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:27.703920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c01cb7-3019-4203-8348-1531d7dc9b2d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:27.728177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:27.740335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fef36-aeba-43a0-8745-431bf865bed6 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:27.741363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:27.742033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:27.742312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:27.742589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:27.752982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:27.755033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:27.770443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:27.772069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:27.773200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:27.773200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:27.773200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:27.774607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 530/1060] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:27.778910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611bdb4-fe96-4151-89ad-70cabbc81568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:27.779965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611bdb4-fe96-4151-89ad-70cabbc81568 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:27.780244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611bdb4-fe96-4151-89ad-70cabbc81568 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:27.780346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:27.780553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611bdb4-fe96-4151-89ad-70cabbc81568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:27.854353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611bdb4-fe96-4151-89ad-70cabbc81568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:27.855805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611bdb4-fe96-4151-89ad-70cabbc81568 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:27.867290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:27.890750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d479f4-de04-45fb-a5e1-baa6b01dba32 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:27.931293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:27.931897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:27.932176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:27.932446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:27.933088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:27.933518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:27.934910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:27.947819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 531/1061] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:27.953295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:27.968590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 531/1062] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:47:27.975252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:28.017161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c3071-5857-4f0e-84e2-7f5459ef247f None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:28.017486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c3071-5857-4f0e-84e2-7f5459ef247f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:28.017713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c3071-5857-4f0e-84e2-7f5459ef247f None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:28.019931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 532/1063] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:28.025432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398c076-8f7b-4e8e-96f1-208dc4b94f1f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:28.025773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398c076-8f7b-4e8e-96f1-208dc4b94f1f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:28.026015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398c076-8f7b-4e8e-96f1-208dc4b94f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:28.046128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2398c076-8f7b-4e8e-96f1-208dc4b94f1f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:28.047966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2398c076-8f7b-4e8e-96f1-208dc4b94f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:28.091319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af9ea2-fd2a-40bf-8e42-2688ebdd2c9c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:28.127007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 533/1064] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:28.130573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:28.131213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:28.131498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:28.131769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:28.132877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] RBAC: Authorizing `identity:get_project(project_id=467936397627429d81f9032347d7e37d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:28.134330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:28.135963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 532/1065] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:47:27 2022] GET /identity/v3/projects/467936397627429d81f9032347d7e37d => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:28.141892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398c076-8f7b-4e8e-96f1-208dc4b94f1f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:28.251114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398c076-8f7b-4e8e-96f1-208dc4b94f1f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:28.269151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:28.287760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:28.288372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:28.288696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:28.288969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:28.289698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=467936397627429d81f9032347d7e37d, user.id=4f89684343e0436b9b66736f008c5c30) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:28.290043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:28.291467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:28.297301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 534/1066] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:28 2022] GET /identity/v3/role_assignments?user.id=4f89684343e0436b9b66736f008c5c30&scope.project.id=467936397627429d81f9032347d7e37d&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:28.406285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:28.415515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:28.510226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2086213-b41f-417b-ac01-255e4872f040 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:28.542345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:28.542971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:28.543257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:28.543530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:28.553949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:28.559132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:28.560240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:28.561137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 533/1067] 10.222.0.72 () {56 vars in 1120 bytes} [Tue Sep 13 12:47:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:28.567385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:28.668461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bbc40-9b4b-416f-9e40-cc5952ff0aff None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:28.703130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:28.703732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:28.704012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:28.704285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:28.704949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:28.705252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:28.707102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:28.715466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 535/1068] 10.222.0.72 () {56 vars in 1115 bytes} [Tue Sep 13 12:47:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:28.721151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:28.785062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:28.818577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2db2fa-7058-4caa-bc15-b4f508157c8a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:28.851035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:28.851651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:28.851928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:28.852200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:28.862309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:28.864500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:28.878286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:28.879396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:28.885315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:28.885315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:28.885315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:28.885315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 534/1069] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:28.886122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:28.931286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:28.985730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7539152-d00c-4789-93bb-6cf5f4342fcb None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:29.020679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:29.021277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:29.021555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:29.021856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:29.022496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:29.022798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:29.024155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:29.035728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 536/1070] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:29.041297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:29.138696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbb20a-433c-4ca1-9626-dc33ea504d78 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:29.156371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:29.174617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:29.175337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:29.175865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:29.176078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:29.183674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:29.185083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:47:29.196448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:29.197543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:47:29.198681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:29.200849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 535/1071] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:47:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:29.207195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:29.318909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bcd30-9241-4420-b26c-d90d61c04177 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:29.321052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:29.352340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:29.352934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:29.353211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:29.353483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:29.354237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:29.354541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:29.355958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:29.372421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 537/1072] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:47:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:29.378247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:29.476469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ffa22-6ec3-421c-b3c4-6cf2ce6ecc8f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:29.508741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:29.509359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:29.509636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:29.509929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:29.520733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:29.522185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:29.535789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:29.536976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:29.538054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:29.538054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:29.538054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:29.541458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 536/1073] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:29.542640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:29.649389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6f6b-35a4-48d2-87e0-c43bb93119fe None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:29.682808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:29.683431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:29.683707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:29.684037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:29.684669 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:29.684965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:29.686345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:29.698136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 538/1074] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:29.704032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:29.735203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:29.802128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7ed38-4557-4c0e-9099-27381801458c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:29.835395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:29.836017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:29.836292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:29.836561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:29.843028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:29.844446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:47:29.855279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:29.856508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:29.857493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:29.858243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 537/1075] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:29.862152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:29.887883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:29.961486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0c367-f6b6-4ab9-8418-2260a1b85ad1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:29.994184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef779ab-6ab9-4262-958d-bdbf4c6c483c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:29.994779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef779ab-6ab9-4262-958d-bdbf4c6c483c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:29.995046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef779ab-6ab9-4262-958d-bdbf4c6c483c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:29.995312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef779ab-6ab9-4262-958d-bdbf4c6c483c None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:29.995992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef779ab-6ab9-4262-958d-bdbf4c6c483c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:29.996290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef779ab-6ab9-4262-958d-bdbf4c6c483c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:29.997702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef779ab-6ab9-4262-958d-bdbf4c6c483c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:30.010664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 539/1076] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:47:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:30.016675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:30.112324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:30.122152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa76719-5503-4b14-8b42-3ffe637d06f0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:30.155876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbc767-5b01-47bd-8a4f-7c0cc8eee7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:30.156454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbc767-5b01-47bd-8a4f-7c0cc8eee7e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:30.156731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbc767-5b01-47bd-8a4f-7c0cc8eee7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:30.157001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbc767-5b01-47bd-8a4f-7c0cc8eee7e5 None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/3b21d991d02b4f95a8d0fcec0a334495/roles/84a9de490add48a7b98e6e4a899e0f30` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:30.159099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbc767-5b01-47bd-8a4f-7c0cc8eee7e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=3b21d991d02b4f95a8d0fcec0a334495, role_id=84a9de490add48a7b98e6e4a899e0f30)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:30.160591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbc767-5b01-47bd-8a4f-7c0cc8eee7e5 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:30.236869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 538/1077] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:47:30 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/3b21d991d02b4f95a8d0fcec0a334495/roles/84a9de490add48a7b98e6e4a899e0f30 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 12:47:30.453208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf192e6e-dbbb-4589-a3f8-28f570cd0829 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:30.453493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf192e6e-dbbb-4589-a3f8-28f570cd0829 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:30.453803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf192e6e-dbbb-4589-a3f8-28f570cd0829 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:30.455156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 540/1078] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:30.461095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a17745-eb84-4b06-b1fb-970b8ed72bfe None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:30.461487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a17745-eb84-4b06-b1fb-970b8ed72bfe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:30.461849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a17745-eb84-4b06-b1fb-970b8ed72bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:30.480201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-45a17745-eb84-4b06-b1fb-970b8ed72bfe None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:30.481368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-45a17745-eb84-4b06-b1fb-970b8ed72bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:30.546943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 539/1079] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:30.815709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf192e6e-dbbb-4589-a3f8-28f570cd0829 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:30.897270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:30.916221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf192e6e-dbbb-4589-a3f8-28f570cd0829 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:30.950553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:30.951178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:30.951456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:30.951729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:30.962420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:30.967219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:30.968343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:30.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 541/1080] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:47:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:30.974579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a17745-eb84-4b06-b1fb-970b8ed72bfe None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:31.051528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:31.070359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a17745-eb84-4b06-b1fb-970b8ed72bfe None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:31.106888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:31.107506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:31.107782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:31.108053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:31.108724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:31.109031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:31.110418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:31.119626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 540/1081] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:47:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:31.125418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:31.223294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b5fb3-875e-4069-bc18-dc9acd138522 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:31.255873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:31.256480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:31.256755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:31.257026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] PATH_INFO: `/v3/users/4f89684343e0436b9b66736f008c5c30` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:31.258159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] RBAC: Authorizing `identity:get_user(user_id=4f89684343e0436b9b66736f008c5c30)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:31.259583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:31.261286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 542/1082] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:31 2022] GET /identity/v3/users/4f89684343e0436b9b66736f008c5c30 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:31.266990 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:31.364558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076a604-18ad-4d6c-ac1e-fc51f92ce74f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:31.397189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590ec6e-c335-48cb-a0e7-690d8ea1446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:31.397840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ec6e-c335-48cb-a0e7-690d8ea1446f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:31.398122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ec6e-c335-48cb-a0e7-690d8ea1446f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:31.398395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590ec6e-c335-48cb-a0e7-690d8ea1446f None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:31.399522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ec6e-c335-48cb-a0e7-690d8ea1446f None admin] RBAC: Authorizing `identity:get_project(project_id=467936397627429d81f9032347d7e37d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:31.400934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590ec6e-c335-48cb-a0e7-690d8ea1446f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:31.402697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 541/1083] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:47:31 2022] GET /identity/v3/projects/467936397627429d81f9032347d7e37d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:31.408796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:31.421836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:31.506828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b722d3f-4927-4fe2-8765-aa43107a49fc None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:31.540448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aac06b-78d2-4101-ba85-245ebea42baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:31.541099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac06b-78d2-4101-ba85-245ebea42baa None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:31.541380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac06b-78d2-4101-ba85-245ebea42baa None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:31.541654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac06b-78d2-4101-ba85-245ebea42baa None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d/users/4f89684343e0436b9b66736f008c5c30/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:31.543565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac06b-78d2-4101-ba85-245ebea42baa None admin] RBAC: Authorizing `identity:create_grant(project_id=467936397627429d81f9032347d7e37d, user_id=4f89684343e0436b9b66736f008c5c30, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:31.544986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac06b-78d2-4101-ba85-245ebea42baa None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:31.612971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 543/1084] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:47:31 2022] PUT /identity/v3/projects/467936397627429d81f9032347d7e37d/users/4f89684343e0436b9b66736f008c5c30/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:47:32.370301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96ef3-2fd9-41d4-9d2e-8b980554f09b None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:32.370867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96ef3-2fd9-41d4-9d2e-8b980554f09b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:32.370867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c96ef3-2fd9-41d4-9d2e-8b980554f09b None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:32.372156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 542/1085] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:32.377581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79c31f-acd1-42a8-842e-6765e1670c78 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:32.378596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79c31f-acd1-42a8-842e-6765e1670c78 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:32.378794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79c31f-acd1-42a8-842e-6765e1670c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:32.397155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-eb79c31f-acd1-42a8-842e-6765e1670c78 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:32.398389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-eb79c31f-acd1-42a8-842e-6765e1670c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:32.459650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:32.464604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 544/1086] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:32.732985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c96ef3-2fd9-41d4-9d2e-8b980554f09b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:32.745576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:32.833962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c96ef3-2fd9-41d4-9d2e-8b980554f09b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:32.867649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:32.868271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:32.868552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:32.868821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:32.879402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:32.884201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:32.885260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:32.886199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 543/1087] 10.222.0.72 () {56 vars in 1120 bytes} [Tue Sep 13 12:47:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:32.891311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79c31f-acd1-42a8-842e-6765e1670c78 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:32.986626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79c31f-acd1-42a8-842e-6765e1670c78 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:33.019121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:33.019717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.019991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.020253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.020897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:33.021189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:33.022574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:33.030127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 545/1088] 10.222.0.72 () {56 vars in 1115 bytes} [Tue Sep 13 12:47:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:33.036558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:33.112180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:33.143065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e1dfc-3614-4883-a493-89ee3223480e None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:33.178733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:33.179509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.179815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.180090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.190050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:33.192165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:33.206553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:33.207612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:33.208954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:33.208954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:33.208954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:33.208954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 544/1089] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:33.213338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:33.273348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:33.311994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7303ca9-5067-4870-8325-1157224276f9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:33.344245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:33.344846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.345118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.345387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.346053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:33.346355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:33.348445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:33.360046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 546/1090] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:33.366442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:33.465570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828385-61b6-4c23-b1fb-d4675ee9a3f2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:33.489011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:33.500223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:33.500850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.501132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.506786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.509500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:33.510961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:47:33.521888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:33.523030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 12:47:33.524198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:33.526362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 545/1091] 10.222.0.72 () {56 vars in 1093 bytes} [Tue Sep 13 12:47:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:33.531931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:33.629884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5523c21-858a-4602-9385-b0f991a68219 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:33.663009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:33.674583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0240ce2-f702-4bfd-af9e-b0fd390384bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:33.677973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0240ce2-f702-4bfd-af9e-b0fd390384bc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.677973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0240ce2-f702-4bfd-af9e-b0fd390384bc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.677973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0240ce2-f702-4bfd-af9e-b0fd390384bc None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.677973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0240ce2-f702-4bfd-af9e-b0fd390384bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:33.677973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0240ce2-f702-4bfd-af9e-b0fd390384bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:33.678492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0240ce2-f702-4bfd-af9e-b0fd390384bc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:33.700199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 547/1092] 10.222.0.72 () {56 vars in 1133 bytes} [Tue Sep 13 12:47:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:33.701474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:33.757899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b894904-e965-421e-af60-2c9dd11c6847 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.758107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b894904-e965-421e-af60-2c9dd11c6847 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.758478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b894904-e965-421e-af60-2c9dd11c6847 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.759721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 548/1093] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:47:33.764689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c0c66-1f05-467e-b8b7-3c38753069d1 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.764970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c0c66-1f05-467e-b8b7-3c38753069d1 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.765410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c0c66-1f05-467e-b8b7-3c38753069d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.783298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-3e8c0c66-1f05-467e-b8b7-3c38753069d1 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:33.785165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3e8c0c66-1f05-467e-b8b7-3c38753069d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:33.799455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6af44-8d7f-427e-8fbe-461b4e9144dc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:33.832589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:33.833184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:33.833464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:33.833759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:33.850940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 549/1094] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:33.881321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:33.899973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:33.901398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:33.994837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 546/1095] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:47:34.000600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c0c66-1f05-467e-b8b7-3c38753069d1 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.098389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c0c66-1f05-467e-b8b7-3c38753069d1 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.108465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:34.118173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.132956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.133586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.133900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.134167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.144726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.146214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:47:34.161062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:34.162226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:47:34.163343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:47:34.163343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:47:34.163343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:34.163343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 550/1096] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:34.168421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.204083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:34.222660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0092640-83b0-4c9b-8ca4-7aa45a2020d1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.256231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.256854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.257135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.257410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.265091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf5918-4b88-489d-8406-10bbeddea2cf None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.267247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:34.272712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:34.273707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:47:34.274632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 547/1097] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:34.279860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.299943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.300536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.300817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.301093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.301730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:34.302069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.303415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.314731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 551/1098] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:34.320772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.344227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:34.377639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a42f02-8e91-4024-8e94-993ae6d8cbb5 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.412122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.412724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.413003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.413279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.413971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:34.414274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.415654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.417244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433bf82-6edd-4814-8cf2-5d56d2f28f9e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.423510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 548/1099] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:34.430486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.433993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:34.454068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.454715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.454989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.455255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.461802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.463309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:47:34.475080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:34.476110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:34.477069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 552/1100] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:47:34.482063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.534837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929db28-e1ce-403d-828c-bf6c827a62e1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.563649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:34.569561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.570310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.570589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.570860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] PATH_INFO: `/v3/users/4f89684343e0436b9b66736f008c5c30` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.571989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] RBAC: Authorizing `identity:get_user(user_id=4f89684343e0436b9b66736f008c5c30)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.573405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.575290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 549/1101] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/users/4f89684343e0436b9b66736f008c5c30 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:34.581138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.581792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57e8ab-f7a0-4af8-91dd-a27b63d5dac9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.617341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.617969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.618253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.618532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.619224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:34.619531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.620955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.633356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 553/1102] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:34.638935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.652059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:34.680966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d1e5f-bb68-47ef-b8c5-8f02d7c019ee None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.714899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.715493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.715770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.716039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.739760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad2a1-d4ed-4277-94d6-3c94d2f6f65b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:34.772432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:34.773036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:34.773315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:34.773597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:34.774345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] RBAC: Adding query filter params (role.id=84a9de490add48a7b98e6e4a899e0f30, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=3b21d991d02b4f95a8d0fcec0a334495) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:34.774659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.776079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.781701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 554/1103] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/role_assignments?user.id=3b21d991d02b4f95a8d0fcec0a334495&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=84a9de490add48a7b98e6e4a899e0f30 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:34.782433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:34.783942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:34.859041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:34.892392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 550/1104] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 12:47:34.899137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:34.909097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:35.005208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3346c45-d8a0-4197-ab9a-e8864763998e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:35.039113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1241e-d47d-40fc-b95f-9e1c7f64ca44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:35.039770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1241e-d47d-40fc-b95f-9e1c7f64ca44 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:35.040049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1241e-d47d-40fc-b95f-9e1c7f64ca44 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:35.040323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1241e-d47d-40fc-b95f-9e1c7f64ca44 None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:35.041461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1241e-d47d-40fc-b95f-9e1c7f64ca44 None admin] RBAC: Authorizing `identity:get_project(project_id=467936397627429d81f9032347d7e37d)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:35.042908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1241e-d47d-40fc-b95f-9e1c7f64ca44 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:35.044541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 555/1105] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:47:34 2022] GET /identity/v3/projects/467936397627429d81f9032347d7e37d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:47:35.051017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:35.154869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4e022-9fa5-4c2a-b3be-f2392d20ae89 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:35.189124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c8e90-58b6-4e39-bde8-927ed79d031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:35.189726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c8e90-58b6-4e39-bde8-927ed79d031c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:35.190028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c8e90-58b6-4e39-bde8-927ed79d031c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:35.190300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c8e90-58b6-4e39-bde8-927ed79d031c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:35.191623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c8e90-58b6-4e39-bde8-927ed79d031c None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=467936397627429d81f9032347d7e37d, user.id=4f89684343e0436b9b66736f008c5c30) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:35.191925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c8e90-58b6-4e39-bde8-927ed79d031c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:35.193383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c8e90-58b6-4e39-bde8-927ed79d031c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:35.199485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 551/1106] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:35 2022] GET /identity/v3/role_assignments?user.id=4f89684343e0436b9b66736f008c5c30&scope.project.id=467936397627429d81f9032347d7e37d&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:37.391549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc91ca-0a29-4ba2-a6bc-6afb7dad7b80 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:37.392113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc91ca-0a29-4ba2-a6bc-6afb7dad7b80 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:37.392113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc91ca-0a29-4ba2-a6bc-6afb7dad7b80 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:37.393974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 556/1107] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:37.399252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15537-81a3-41da-bbf0-e3e199d3f678 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:37.399673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15537-81a3-41da-bbf0-e3e199d3f678 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:37.399797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b15537-81a3-41da-bbf0-e3e199d3f678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:37.417987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c4b15537-81a3-41da-bbf0-e3e199d3f678 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:37.419151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c4b15537-81a3-41da-bbf0-e3e199d3f678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:37.446712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:37.485229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 552/1108] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:47:37.751515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc91ca-0a29-4ba2-a6bc-6afb7dad7b80 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:37.825810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:37.851118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc91ca-0a29-4ba2-a6bc-6afb7dad7b80 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:37.884350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcbaf9-44a2-47cf-b2c9-e3ea67cd38f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:37.884958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbaf9-44a2-47cf-b2c9-e3ea67cd38f8 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:37.885234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbaf9-44a2-47cf-b2c9-e3ea67cd38f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:37.885505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcbaf9-44a2-47cf-b2c9-e3ea67cd38f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:37.886681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcbaf9-44a2-47cf-b2c9-e3ea67cd38f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:37.888074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcbaf9-44a2-47cf-b2c9-e3ea67cd38f8 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:37.889620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 557/1109] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:37.896065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b15537-81a3-41da-bbf0-e3e199d3f678 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:37.992178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b15537-81a3-41da-bbf0-e3e199d3f678 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:38.025501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e978740-913c-4249-bae6-13341e14d915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:38.026070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978740-913c-4249-bae6-13341e14d915 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:38.026347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978740-913c-4249-bae6-13341e14d915 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:38.026620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978740-913c-4249-bae6-13341e14d915 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:38.027260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e978740-913c-4249-bae6-13341e14d915 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:38.028679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e978740-913c-4249-bae6-13341e14d915 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:38.128865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 553/1110] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:47:37 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:40.292094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba84c4-9752-4522-8d64-0a4b09bfa1ee None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:40.292623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba84c4-9752-4522-8d64-0a4b09bfa1ee None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:40.292623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba84c4-9752-4522-8d64-0a4b09bfa1ee None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:40.294011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 558/1111] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:40.299347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b2c8b-d0df-4280-a7e1-9679465c6f99 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:40.299862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b2c8b-d0df-4280-a7e1-9679465c6f99 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:40.300146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b2c8b-d0df-4280-a7e1-9679465c6f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:40.319166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a99b2c8b-d0df-4280-a7e1-9679465c6f99 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:40.320345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a99b2c8b-d0df-4280-a7e1-9679465c6f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:40.337107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:40.386312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 554/1112] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:40.658776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba84c4-9752-4522-8d64-0a4b09bfa1ee None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:40.758861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba84c4-9752-4522-8d64-0a4b09bfa1ee None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:40.795284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:40.795918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:40.796195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:40.796474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] PATH_INFO: `/v3/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:40.797869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] RBAC: Authorizing `identity:get_role(role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:40.799234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:40.801201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 559/1113] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:40 2022] GET /identity/v3/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:40.808485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b2c8b-d0df-4280-a7e1-9679465c6f99 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:40.908448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b2c8b-d0df-4280-a7e1-9679465c6f99 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:40.943291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:40.943922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:40.944202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:40.944479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] PATH_INFO: `/v3/users/9a4abec40fdb4286816ce4b57a17e52d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:40.958754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] RBAC: Authorizing `identity:get_user(user_id=9a4abec40fdb4286816ce4b57a17e52d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:40.960819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:40.961848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 555/1114] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:40 2022] GET /identity/v3/users/9a4abec40fdb4286816ce4b57a17e52d => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:40.967704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:40.980369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:41.076933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d615c-886c-4856-8c11-15fa664c87c7 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:41.110210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:41.110809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:41.111087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:41.111358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:41.180217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:41.181633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:41.198404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:41.285374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 560/1115] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:47:41.291847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:41.304970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:41.392627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7a94d-9ed9-4fe1-bdfc-ca639b231d3d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:41.427452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fa2f8-c938-46fe-a9c5-b94b168b7f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:41.428083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fa2f8-c938-46fe-a9c5-b94b168b7f44 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:41.428364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fa2f8-c938-46fe-a9c5-b94b168b7f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:41.428638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fa2f8-c938-46fe-a9c5-b94b168b7f44 None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:41.429781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fa2f8-c938-46fe-a9c5-b94b168b7f44 None admin] RBAC: Authorizing `identity:get_project(project_id=467936397627429d81f9032347d7e37d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:41.431262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229fa2f8-c938-46fe-a9c5-b94b168b7f44 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:41.432844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 556/1116] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:47:41 2022] GET /identity/v3/projects/467936397627429d81f9032347d7e37d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:41.439079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:41.535622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b92a75-f1b1-456c-8702-ccd94aec2660 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:41.545785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:41.569156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97da084-89ba-4e9b-a975-7d9660664613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:41.569773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97da084-89ba-4e9b-a975-7d9660664613 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:41.570347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97da084-89ba-4e9b-a975-7d9660664613 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:41.570618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97da084-89ba-4e9b-a975-7d9660664613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:41.571403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97da084-89ba-4e9b-a975-7d9660664613 None admin] RBAC: Adding query filter params (role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=467936397627429d81f9032347d7e37d, user.id=9a4abec40fdb4286816ce4b57a17e52d) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:41.571707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97da084-89ba-4e9b-a975-7d9660664613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:41.573115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97da084-89ba-4e9b-a975-7d9660664613 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:41.579245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 561/1117] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:41 2022] GET /identity/v3/role_assignments?user.id=9a4abec40fdb4286816ce4b57a17e52d&scope.project.id=467936397627429d81f9032347d7e37d&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:43.715756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a503b4e-9467-4d8a-a922-391fb74885d1 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:43.716258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a503b4e-9467-4d8a-a922-391fb74885d1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:43.716369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a503b4e-9467-4d8a-a922-391fb74885d1 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:43.717619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 557/1118] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:43.723225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45fdb5-7146-4e69-b696-6189806c020c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:43.723620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45fdb5-7146-4e69-b696-6189806c020c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:43.723761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45fdb5-7146-4e69-b696-6189806c020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:43.742167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9a45fdb5-7146-4e69-b696-6189806c020c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:43.743351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9a45fdb5-7146-4e69-b696-6189806c020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:43.806919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 562/1119] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:44.077710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a503b4e-9467-4d8a-a922-391fb74885d1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:44.154815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:44.176901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a503b4e-9467-4d8a-a922-391fb74885d1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:44.209843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:44.210460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:44.210741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:44.211014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] PATH_INFO: `/v3/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:44.212398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] RBAC: Authorizing `identity:get_role(role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:44.213769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:44.215074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 558/1120] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:44 2022] GET /identity/v3/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:44.220824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45fdb5-7146-4e69-b696-6189806c020c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:44.315462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45fdb5-7146-4e69-b696-6189806c020c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:44.317867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:44.349465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:44.350051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:44.350330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:44.350601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] PATH_INFO: `/v3/users/9a4abec40fdb4286816ce4b57a17e52d` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:44.351709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] RBAC: Authorizing `identity:get_user(user_id=9a4abec40fdb4286816ce4b57a17e52d)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:44.353128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:44.354720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 563/1121] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:44 2022] GET /identity/v3/users/9a4abec40fdb4286816ce4b57a17e52d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:44.360477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:44.456738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cdfa26-70b1-4570-826d-014aadc5f6c8 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:44.489029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47875c1-f5bd-4a25-8805-59d167d8e00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:44.489646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47875c1-f5bd-4a25-8805-59d167d8e00a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:44.489951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47875c1-f5bd-4a25-8805-59d167d8e00a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:44.490225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47875c1-f5bd-4a25-8805-59d167d8e00a None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:44.491321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47875c1-f5bd-4a25-8805-59d167d8e00a None admin] RBAC: Authorizing `identity:get_project(project_id=467936397627429d81f9032347d7e37d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:44.492733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47875c1-f5bd-4a25-8805-59d167d8e00a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:44.494358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 559/1122] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:47:44 2022] GET /identity/v3/projects/467936397627429d81f9032347d7e37d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:44.500177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:44.611789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7afae-ac77-406f-b909-561162bcb9b4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:44.646007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91b966-83bb-40d4-8f48-f64fe084acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:44.646668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b966-83bb-40d4-8f48-f64fe084acb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:44.646947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b966-83bb-40d4-8f48-f64fe084acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:44.647219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b966-83bb-40d4-8f48-f64fe084acb5 None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d/users/9a4abec40fdb4286816ce4b57a17e52d/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:44.649310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91b966-83bb-40d4-8f48-f64fe084acb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=467936397627429d81f9032347d7e37d, user_id=9a4abec40fdb4286816ce4b57a17e52d, role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:44.650766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91b966-83bb-40d4-8f48-f64fe084acb5 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:44.704479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 564/1123] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:47:44 2022] PUT /identity/v3/projects/467936397627429d81f9032347d7e37d/users/9a4abec40fdb4286816ce4b57a17e52d/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:47:46.821361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e61d-ad0d-4cd1-a277-3dcd84f7fa65 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:46.821906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e61d-ad0d-4cd1-a277-3dcd84f7fa65 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:46.821906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e61d-ad0d-4cd1-a277-3dcd84f7fa65 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:46.823259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 560/1124] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:46.828608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82ecdf-ab07-4799-a32b-57103b7dc2a4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:46.828987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82ecdf-ab07-4799-a32b-57103b7dc2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:46.829148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82ecdf-ab07-4799-a32b-57103b7dc2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:46.848144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ef82ecdf-ab07-4799-a32b-57103b7dc2a4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:46.849319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ef82ecdf-ab07-4799-a32b-57103b7dc2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:46.873855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:46.914629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 565/1125] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:47.181859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2e61d-ad0d-4cd1-a277-3dcd84f7fa65 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:47.197691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:47.280716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2e61d-ad0d-4cd1-a277-3dcd84f7fa65 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:47.314869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:47.316046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:47.316323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:47.316597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] PATH_INFO: `/v3/roles/7ff495d40d2b46c4bf149b6ecaec6996` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:47.318002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] RBAC: Authorizing `identity:get_role(role_id=7ff495d40d2b46c4bf149b6ecaec6996)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:47.319349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:47.320649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 561/1126] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:47 2022] GET /identity/v3/roles/7ff495d40d2b46c4bf149b6ecaec6996 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:47.326754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82ecdf-ab07-4799-a32b-57103b7dc2a4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:47.426355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82ecdf-ab07-4799-a32b-57103b7dc2a4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:47.458844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:47.459457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:47.459734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:47.460004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] PATH_INFO: `/v3/users/9a4abec40fdb4286816ce4b57a17e52d` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:47.461108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] RBAC: Authorizing `identity:get_user(user_id=9a4abec40fdb4286816ce4b57a17e52d)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:47.462557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:47.464110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 566/1127] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:47 2022] GET /identity/v3/users/9a4abec40fdb4286816ce4b57a17e52d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:47.469693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:47.548099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:47.565842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05aed58-7710-4855-9684-b1f2434bea58 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:47.599187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:47.599780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:47.600060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:47.600333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:47.664518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:47.665970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:47.743246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:47.760555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 562/1128] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:47:47.766467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:47.796453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:47.864591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23580722-9358-4b7c-8089-8fae741f4258 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:47.897054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a171c1-546c-40d6-b91d-f207584c05f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:47.897676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a171c1-546c-40d6-b91d-f207584c05f7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:47.897981 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a171c1-546c-40d6-b91d-f207584c05f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:47.898255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a171c1-546c-40d6-b91d-f207584c05f7 None admin] PATH_INFO: `/v3/projects/467936397627429d81f9032347d7e37d` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:47.899353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a171c1-546c-40d6-b91d-f207584c05f7 None admin] RBAC: Authorizing `identity:get_project(project_id=467936397627429d81f9032347d7e37d)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:47.900775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a171c1-546c-40d6-b91d-f207584c05f7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:47.902406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 567/1129] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:47:47 2022] GET /identity/v3/projects/467936397627429d81f9032347d7e37d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:47.908441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:48.003150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cf8ef-82ad-4020-b168-b17491d5331f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:48.035883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e618c-b0ee-4752-a4f9-a7c69993ed72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:48.036471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e618c-b0ee-4752-a4f9-a7c69993ed72 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:48.036748 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e618c-b0ee-4752-a4f9-a7c69993ed72 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:48.037017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e618c-b0ee-4752-a4f9-a7c69993ed72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:48.037763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e618c-b0ee-4752-a4f9-a7c69993ed72 None admin] RBAC: Adding query filter params (role.id=7ff495d40d2b46c4bf149b6ecaec6996, scope.project.id=467936397627429d81f9032347d7e37d, user.id=9a4abec40fdb4286816ce4b57a17e52d) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:48.038066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e618c-b0ee-4752-a4f9-a7c69993ed72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:48.039468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e618c-b0ee-4752-a4f9-a7c69993ed72 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:48.044970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 563/1130] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:47 2022] GET /identity/v3/role_assignments?user.id=9a4abec40fdb4286816ce4b57a17e52d&scope.project.id=467936397627429d81f9032347d7e37d&role.id=7ff495d40d2b46c4bf149b6ecaec6996 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:50.172963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0334-0ee1-402d-b21f-63b44431c99b None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:50.173534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0334-0ee1-402d-b21f-63b44431c99b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:50.173534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de0334-0ee1-402d-b21f-63b44431c99b None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:50.175639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 568/1131] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:50.180996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4d950-84eb-432f-8759-13091ec9b872 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:50.181380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4d950-84eb-432f-8759-13091ec9b872 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:50.181542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4d950-84eb-432f-8759-13091ec9b872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:50.200419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-57e4d950-84eb-432f-8759-13091ec9b872 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:50.201588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-57e4d950-84eb-432f-8759-13091ec9b872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:50.241764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:50.265028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 564/1132] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:50.534014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de0334-0ee1-402d-b21f-63b44431c99b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:50.626351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:50.631478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de0334-0ee1-402d-b21f-63b44431c99b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:50.666027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe82c99-f146-4749-82c9-500faf776467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:50.666647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe82c99-f146-4749-82c9-500faf776467 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:50.666927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe82c99-f146-4749-82c9-500faf776467 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:50.667197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe82c99-f146-4749-82c9-500faf776467 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:50.668309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe82c99-f146-4749-82c9-500faf776467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:50.669718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe82c99-f146-4749-82c9-500faf776467 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:50.671288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 569/1133] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:50.677116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4d950-84eb-432f-8759-13091ec9b872 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:50.771792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4d950-84eb-432f-8759-13091ec9b872 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:50.803648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d44ec2-697a-4e4f-bb54-03795e268d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:50.804189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44ec2-697a-4e4f-bb54-03795e268d71 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:50.804469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44ec2-697a-4e4f-bb54-03795e268d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:50.804744 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44ec2-697a-4e4f-bb54-03795e268d71 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:50.805388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d44ec2-697a-4e4f-bb54-03795e268d71 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:50.806839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d44ec2-697a-4e4f-bb54-03795e268d71 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:50.917245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 565/1134] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:47:50 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:53.092208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082f6c3-d5c8-47d7-b188-fcbd78fea385 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:53.092774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082f6c3-d5c8-47d7-b188-fcbd78fea385 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:53.092774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082f6c3-d5c8-47d7-b188-fcbd78fea385 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:53.094115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 570/1135] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:53.100137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f12bec-0b11-4c51-9488-b20b1684abb9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:53.100503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f12bec-0b11-4c51-9488-b20b1684abb9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:53.100693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f12bec-0b11-4c51-9488-b20b1684abb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:53.119310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-08f12bec-0b11-4c51-9488-b20b1684abb9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:53.120486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-08f12bec-0b11-4c51-9488-b20b1684abb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:53.142286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:53.185578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 566/1136] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:53.460133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082f6c3-d5c8-47d7-b188-fcbd78fea385 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:53.558728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082f6c3-d5c8-47d7-b188-fcbd78fea385 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:53.591213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277e0cc-c72e-4d9c-a472-54d6279583b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:53.591831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277e0cc-c72e-4d9c-a472-54d6279583b2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:53.592109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277e0cc-c72e-4d9c-a472-54d6279583b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:53.592381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277e0cc-c72e-4d9c-a472-54d6279583b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:53.593928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277e0cc-c72e-4d9c-a472-54d6279583b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:53.595479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277e0cc-c72e-4d9c-a472-54d6279583b2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:53.597828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 571/1137] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:47:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:53.603545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f12bec-0b11-4c51-9488-b20b1684abb9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:53.699747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f12bec-0b11-4c51-9488-b20b1684abb9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:53.732301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221a6e-93dc-40e2-b178-5f9c2a03ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:53.732847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221a6e-93dc-40e2-b178-5f9c2a03ba67 None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:53.733128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221a6e-93dc-40e2-b178-5f9c2a03ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:53.733401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221a6e-93dc-40e2-b178-5f9c2a03ba67 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:53.734083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5221a6e-93dc-40e2-b178-5f9c2a03ba67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:53.735500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5221a6e-93dc-40e2-b178-5f9c2a03ba67 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:53.870188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 567/1138] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:47:53 2022] POST /identity/v3/users => generated 286 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:47:56.034503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dfe83-2688-4f0a-ae88-42a9a28930d5 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:56.035089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dfe83-2688-4f0a-ae88-42a9a28930d5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:56.035241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dfe83-2688-4f0a-ae88-42a9a28930d5 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:56.036648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 572/1139] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:56.042133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493502cc-2f6c-4b6b-a85e-a7f418705ce1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:56.042551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493502cc-2f6c-4b6b-a85e-a7f418705ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:56.042687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493502cc-2f6c-4b6b-a85e-a7f418705ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:56.060858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-493502cc-2f6c-4b6b-a85e-a7f418705ce1 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:56.062065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-493502cc-2f6c-4b6b-a85e-a7f418705ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:56.071086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:56.128427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 568/1140] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:56.393439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dfe83-2688-4f0a-ae88-42a9a28930d5 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:56.428947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:56.494036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dfe83-2688-4f0a-ae88-42a9a28930d5 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:56.525808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:56.526420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:56.526694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:56.526962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:56.537362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:47:56.542110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:47:56.543205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:47:56.544253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 573/1141] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:47:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:47:56.550105 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493502cc-2f6c-4b6b-a85e-a7f418705ce1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:56.644063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493502cc-2f6c-4b6b-a85e-a7f418705ce1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:56.676959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:56.677547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:56.677847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:56.678120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:56.678771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:56.679070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:56.680417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:56.688200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 569/1142] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:47:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:56.694334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:56.784485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:56.790945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d428484-5c6f-43d1-9b7e-c38016db737c None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:56.824300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:56.824906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:56.825180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:56.825446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] PATH_INFO: `/v3/users/de98a6c50aa04e77aeac9f15d82f5e53` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:56.838525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] RBAC: Authorizing `identity:get_user(user_id=de98a6c50aa04e77aeac9f15d82f5e53)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:56.839946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:56.841497 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 574/1143] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:47:56 2022] GET /identity/v3/users/de98a6c50aa04e77aeac9f15d82f5e53 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:56.847562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:56.856561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:56.946166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95240e5-6988-49f9-be25-409fd6adeb3c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:56.991270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:56.991865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:56.992143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:56.992414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:57.056186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:57.057606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:57.064795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:57.157689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 570/1144] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:47:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:47:57.163970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:57.264383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f07e6-4d5b-4df4-91c4-3a12498cceb0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:57.296906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37914a18-5bb1-49bc-ac10-5240194b4420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:57.297531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914a18-5bb1-49bc-ac10-5240194b4420 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:57.297840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914a18-5bb1-49bc-ac10-5240194b4420 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:57.298116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37914a18-5bb1-49bc-ac10-5240194b4420 None admin] PATH_INFO: `/v3/projects/970b6a392eaf438e9d36b261b3b90b2b` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:57.299850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37914a18-5bb1-49bc-ac10-5240194b4420 None admin] RBAC: Authorizing `identity:get_project(project_id=970b6a392eaf438e9d36b261b3b90b2b)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:57.301469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37914a18-5bb1-49bc-ac10-5240194b4420 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:57.303115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 575/1145] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:47:57 2022] GET /identity/v3/projects/970b6a392eaf438e9d36b261b3b90b2b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:57.309229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:57.405062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e652e-6ea6-4498-b0e5-3f19687d1fbf None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:47:57.412618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:47:57.438199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d61516-6a9f-43da-bcd5-f24f5f5c8539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:47:57.438797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61516-6a9f-43da-bcd5-f24f5f5c8539 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:57.439077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61516-6a9f-43da-bcd5-f24f5f5c8539 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:57.439350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d61516-6a9f-43da-bcd5-f24f5f5c8539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:57.440239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d61516-6a9f-43da-bcd5-f24f5f5c8539 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=970b6a392eaf438e9d36b261b3b90b2b, user.id=de98a6c50aa04e77aeac9f15d82f5e53) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:47:57.440623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d61516-6a9f-43da-bcd5-f24f5f5c8539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:47:57.442125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d61516-6a9f-43da-bcd5-f24f5f5c8539 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:47:57.447509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 571/1146] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:47:57 2022] GET /identity/v3/role_assignments?user.id=de98a6c50aa04e77aeac9f15d82f5e53&scope.project.id=970b6a392eaf438e9d36b261b3b90b2b&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:47:59.547026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55d67-e684-46dc-b063-7a7feb4d7760 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:59.547583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55d67-e684-46dc-b063-7a7feb4d7760 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:59.547583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a55d67-e684-46dc-b063-7a7feb4d7760 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:59.548888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 576/1147] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:47:59.554256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3efd3d-1e62-4ca5-bbe4-055ba7d707aa None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:47:59.554673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3efd3d-1e62-4ca5-bbe4-055ba7d707aa None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:47:59.554807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3efd3d-1e62-4ca5-bbe4-055ba7d707aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:47:59.572861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3f3efd3d-1e62-4ca5-bbe4-055ba7d707aa None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:47:59.574335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3f3efd3d-1e62-4ca5-bbe4-055ba7d707aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:47:59.637865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 572/1148] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:47:59.904211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55d67-e684-46dc-b063-7a7feb4d7760 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:47:59.926031 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:00.003631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a55d67-e684-46dc-b063-7a7feb4d7760 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:00.036676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:00.037287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:00.037566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:00.037863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:00.047550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:48:00.052469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:48:00.053447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:48:00.054418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 577/1149] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:48:00.059603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3efd3d-1e62-4ca5-bbe4-055ba7d707aa None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:00.150878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:00.153666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3efd3d-1e62-4ca5-bbe4-055ba7d707aa None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:00.185905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:00.186439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:00.186717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:00.186986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:00.187668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:00.187970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:00.189314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:00.198241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 573/1150] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:48:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:00.204637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:00.283044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:00.301287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa4390-110f-4126-b584-ca5c941d5c7e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:00.336924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:00.337545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:00.337852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:00.338127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] PATH_INFO: `/v3/users/de98a6c50aa04e77aeac9f15d82f5e53` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:00.339262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] RBAC: Authorizing `identity:get_user(user_id=de98a6c50aa04e77aeac9f15d82f5e53)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:00.340766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:00.342425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 578/1151] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:48:00 2022] GET /identity/v3/users/de98a6c50aa04e77aeac9f15d82f5e53 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:00.348538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:00.443938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc07bbc-d9e0-40ce-883e-c1957258653f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:00.477818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94f95e-3709-49c2-a08b-60b9b3053bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:00.478436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f95e-3709-49c2-a08b-60b9b3053bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:00.478713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f95e-3709-49c2-a08b-60b9b3053bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:00.478984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f95e-3709-49c2-a08b-60b9b3053bc2 None admin] PATH_INFO: `/v3/projects/970b6a392eaf438e9d36b261b3b90b2b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:00.480072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94f95e-3709-49c2-a08b-60b9b3053bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=970b6a392eaf438e9d36b261b3b90b2b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:00.481481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94f95e-3709-49c2-a08b-60b9b3053bc2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:00.483117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 574/1152] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:48:00 2022] GET /identity/v3/projects/970b6a392eaf438e9d36b261b3b90b2b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:00.489284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:00.584662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73bb5-e552-41ba-b91a-23abc4053c7f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:00.618041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d00b7-8a62-4cd9-b051-3cc82da796dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:00.618682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d00b7-8a62-4cd9-b051-3cc82da796dc None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:00.618959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d00b7-8a62-4cd9-b051-3cc82da796dc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:00.619231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d00b7-8a62-4cd9-b051-3cc82da796dc None admin] PATH_INFO: `/v3/projects/970b6a392eaf438e9d36b261b3b90b2b/users/de98a6c50aa04e77aeac9f15d82f5e53/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:00.621079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d00b7-8a62-4cd9-b051-3cc82da796dc None admin] RBAC: Authorizing `identity:create_grant(project_id=970b6a392eaf438e9d36b261b3b90b2b, user_id=de98a6c50aa04e77aeac9f15d82f5e53, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:00.622531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d00b7-8a62-4cd9-b051-3cc82da796dc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:00.704915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 579/1153] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:48:00 2022] PUT /identity/v3/projects/970b6a392eaf438e9d36b261b3b90b2b/users/de98a6c50aa04e77aeac9f15d82f5e53/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:48:02.830963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52b7ef-8252-4f12-ac74-ebb97ca5c978 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:02.831507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52b7ef-8252-4f12-ac74-ebb97ca5c978 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:02.831607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52b7ef-8252-4f12-ac74-ebb97ca5c978 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:02.832911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 575/1154] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:02.838351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb99e1-de3c-4579-9fba-9cbc81168169 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:02.838767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb99e1-de3c-4579-9fba-9cbc81168169 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:02.838876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb99e1-de3c-4579-9fba-9cbc81168169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:02.857068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4efb99e1-de3c-4579-9fba-9cbc81168169 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:02.858275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4efb99e1-de3c-4579-9fba-9cbc81168169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:02.860180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:02.921925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 580/1155] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:03.191313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52b7ef-8252-4f12-ac74-ebb97ca5c978 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:03.221450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:03.290352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52b7ef-8252-4f12-ac74-ebb97ca5c978 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:03.323308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:03.324186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:03.324459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:03.324724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:03.335366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:48:03.340086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:48:03.341295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:48:03.342427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 576/1156] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:48:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:48:03.347338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb99e1-de3c-4579-9fba-9cbc81168169 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:03.442250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb99e1-de3c-4579-9fba-9cbc81168169 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:03.475126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:03.475729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:03.476006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:03.476279 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:03.476937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:03.477237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:03.478619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:03.483469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:03.486614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 581/1157] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:48:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:03.492496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:03.582660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:03.591706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0cef-672e-4a64-a430-a3e0cf5dd06c None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:03.624645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:03.625252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:03.625528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:03.625822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] PATH_INFO: `/v3/users/de98a6c50aa04e77aeac9f15d82f5e53` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:03.626917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] RBAC: Authorizing `identity:get_user(user_id=de98a6c50aa04e77aeac9f15d82f5e53)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:03.628883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:03.630469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 577/1158] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:48:03 2022] GET /identity/v3/users/de98a6c50aa04e77aeac9f15d82f5e53 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:03.636261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:03.734480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef59b13-ce5c-483d-af37-bd0d54b0c198 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:03.766686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:03.767277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:03.767551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:03.767869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:03.829504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:03.831724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:03.833152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:03.926175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 582/1159] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:48:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:48:03.932222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:04.026285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2e0ba-4b52-4a35-8bc7-cf2151603d95 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:04.058586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069c9d9-a596-43eb-923c-f307e3ec8b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:04.059189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c9d9-a596-43eb-923c-f307e3ec8b04 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:04.059466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c9d9-a596-43eb-923c-f307e3ec8b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:04.059735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c9d9-a596-43eb-923c-f307e3ec8b04 None admin] PATH_INFO: `/v3/projects/970b6a392eaf438e9d36b261b3b90b2b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:04.060811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069c9d9-a596-43eb-923c-f307e3ec8b04 None admin] RBAC: Authorizing `identity:get_project(project_id=970b6a392eaf438e9d36b261b3b90b2b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:04.062245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069c9d9-a596-43eb-923c-f307e3ec8b04 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:04.063830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 578/1160] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:48:03 2022] GET /identity/v3/projects/970b6a392eaf438e9d36b261b3b90b2b => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:04.070388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:04.164318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:04.167149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731db6d-57f2-4eeb-864c-56be8eb3ae8e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:04.205089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebba466-093e-4a4c-bd2e-0bc957b3d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:04.206440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebba466-093e-4a4c-bd2e-0bc957b3d88c None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:04.206720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebba466-093e-4a4c-bd2e-0bc957b3d88c None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:04.206993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebba466-093e-4a4c-bd2e-0bc957b3d88c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:04.207707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebba466-093e-4a4c-bd2e-0bc957b3d88c None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=970b6a392eaf438e9d36b261b3b90b2b, user.id=de98a6c50aa04e77aeac9f15d82f5e53) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:04.208014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebba466-093e-4a4c-bd2e-0bc957b3d88c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:04.209430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebba466-093e-4a4c-bd2e-0bc957b3d88c None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:04.215071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 583/1161] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:48:04 2022] GET /identity/v3/role_assignments?user.id=de98a6c50aa04e77aeac9f15d82f5e53&scope.project.id=970b6a392eaf438e9d36b261b3b90b2b&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:06.366540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e90dcc-9e84-4a96-9967-814c93b7e776 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:06.367095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e90dcc-9e84-4a96-9967-814c93b7e776 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:06.367095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e90dcc-9e84-4a96-9967-814c93b7e776 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:06.368418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 579/1162] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:06.373825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d7439-a927-4ccd-94ba-21104e20e943 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:06.374244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d7439-a927-4ccd-94ba-21104e20e943 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:06.374380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d7439-a927-4ccd-94ba-21104e20e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:06.392946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2f0d7439-a927-4ccd-94ba-21104e20e943 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:06.394274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2f0d7439-a927-4ccd-94ba-21104e20e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:06.458942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 584/1163] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:06.730485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e90dcc-9e84-4a96-9967-814c93b7e776 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:06.757139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:06.828345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e90dcc-9e84-4a96-9967-814c93b7e776 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:06.861257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:06.862078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:06.862359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:06.862629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:06.926600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:06.927972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:06.953184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:07.025250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 580/1164] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:48:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:48:07.031199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d7439-a927-4ccd-94ba-21104e20e943 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:07.120156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:07.126621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d7439-a927-4ccd-94ba-21104e20e943 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:07.160106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37722024-97c5-4c09-ae0d-3f5971f8744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:07.160717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722024-97c5-4c09-ae0d-3f5971f8744a None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:07.160999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722024-97c5-4c09-ae0d-3f5971f8744a None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:07.161275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722024-97c5-4c09-ae0d-3f5971f8744a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:07.167716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37722024-97c5-4c09-ae0d-3f5971f8744a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:07.169157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37722024-97c5-4c09-ae0d-3f5971f8744a None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-37722024-97c5-4c09-ae0d-3f5971f8744a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:48:07.180588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:48:07.181786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:48:07.182658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 585/1165] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:48:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:48:07.187693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:07.285217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092b2c1-d984-4f63-a5a0-e16f08e2dc7b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:07.307283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:07.316310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc6a73-3b46-4c07-83c9-f61c944552a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:07.317788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6a73-3b46-4c07-83c9-f61c944552a9 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:07.317788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6a73-3b46-4c07-83c9-f61c944552a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:07.317788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc6a73-3b46-4c07-83c9-f61c944552a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:07.318151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc6a73-3b46-4c07-83c9-f61c944552a9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:07.318453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc6a73-3b46-4c07-83c9-f61c944552a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:07.319813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc6a73-3b46-4c07-83c9-f61c944552a9 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:07.327195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 581/1166] 10.222.0.72 () {56 vars in 1115 bytes} [Tue Sep 13 12:48:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:09.456413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743e4d4-699f-4deb-8ec0-0aadd97238d6 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:09.456945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743e4d4-699f-4deb-8ec0-0aadd97238d6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:09.456945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743e4d4-699f-4deb-8ec0-0aadd97238d6 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:09.458312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 586/1167] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:09.463531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12d31-d522-4fb0-8f68-34d650c89084 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:09.463914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12d31-d522-4fb0-8f68-34d650c89084 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:09.464081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12d31-d522-4fb0-8f68-34d650c89084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:09.482437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bbe12d31-d522-4fb0-8f68-34d650c89084 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:09.483603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bbe12d31-d522-4fb0-8f68-34d650c89084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:09.547213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 582/1168] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:09.821316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743e4d4-699f-4deb-8ec0-0aadd97238d6 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:09.918994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743e4d4-699f-4deb-8ec0-0aadd97238d6 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:09.950774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b4146-62b6-4341-8e42-bf1c3e61f1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:09.951360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b4146-62b6-4341-8e42-bf1c3e61f1fe None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:09.951640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b4146-62b6-4341-8e42-bf1c3e61f1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:09.951910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b4146-62b6-4341-8e42-bf1c3e61f1fe None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:09.952604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b4146-62b6-4341-8e42-bf1c3e61f1fe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:09.953980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b4146-62b6-4341-8e42-bf1c3e61f1fe None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:10.088813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 587/1169] 10.222.0.72 () {60 vars in 1138 bytes} [Tue Sep 13 12:48:09 2022] POST /identity/v3/domains => generated 267 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:48:12.249196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f650-01be-4af3-827a-29eb1b1b3ee2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:12.249787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f650-01be-4af3-827a-29eb1b1b3ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:12.249787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f650-01be-4af3-827a-29eb1b1b3ee2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:12.251117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 583/1170] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:12.256470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528d86-c114-4f43-bb44-542f0cd48ae3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:12.256864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528d86-c114-4f43-bb44-542f0cd48ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:12.257020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528d86-c114-4f43-bb44-542f0cd48ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:12.275037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6a528d86-c114-4f43-bb44-542f0cd48ae3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:12.276204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6a528d86-c114-4f43-bb44-542f0cd48ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:12.285987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:12.340261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 588/1171] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:12.611159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749f650-01be-4af3-827a-29eb1b1b3ee2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:12.709349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749f650-01be-4af3-827a-29eb1b1b3ee2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:12.729228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:12.743644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b335e2-c61e-4372-8d3b-d862517e3046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:12.744267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b335e2-c61e-4372-8d3b-d862517e3046 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:12.744548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b335e2-c61e-4372-8d3b-d862517e3046 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:12.744821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b335e2-c61e-4372-8d3b-d862517e3046 None admin] PATH_INFO: `/v3/domains/de115f74853d4a96a40b33fc04c85607` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:12.755473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b335e2-c61e-4372-8d3b-d862517e3046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=de115f74853d4a96a40b33fc04c85607)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:12.756903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b335e2-c61e-4372-8d3b-d862517e3046 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:12.758515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 584/1172] 10.222.0.72 () {56 vars in 1183 bytes} [Tue Sep 13 12:48:12 2022] GET /identity/v3/domains/de115f74853d4a96a40b33fc04c85607 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:12.764319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528d86-c114-4f43-bb44-542f0cd48ae3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:12.860189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528d86-c114-4f43-bb44-542f0cd48ae3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:12.893421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9a6e5-bf65-446f-ad8d-b1c758886328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:12.893972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a6e5-bf65-446f-ad8d-b1c758886328 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:12.894249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a6e5-bf65-446f-ad8d-b1c758886328 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:12.894520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a6e5-bf65-446f-ad8d-b1c758886328 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:12.895173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9a6e5-bf65-446f-ad8d-b1c758886328 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:12.896630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9a6e5-bf65-446f-ad8d-b1c758886328 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:12.912187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:13.013446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 589/1173] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:48:12 2022] POST /identity/v3/projects => generated 361 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:48:15.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112ec2-5444-488d-a095-7bd913ff228b None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:15.172312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112ec2-5444-488d-a095-7bd913ff228b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:15.172312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23112ec2-5444-488d-a095-7bd913ff228b None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:15.174184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 585/1174] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:15.179554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17b425-1573-43e1-9d8c-f2b0be7dcad4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:15.179947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17b425-1573-43e1-9d8c-f2b0be7dcad4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:15.180098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17b425-1573-43e1-9d8c-f2b0be7dcad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:15.198612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6b17b425-1573-43e1-9d8c-f2b0be7dcad4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:15.199779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6b17b425-1573-43e1-9d8c-f2b0be7dcad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:15.264080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 590/1175] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:15.537102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23112ec2-5444-488d-a095-7bd913ff228b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:15.635651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23112ec2-5444-488d-a095-7bd913ff228b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:15.669436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554fd01-6878-4f16-8c4f-21710b72c5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:15.670082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554fd01-6878-4f16-8c4f-21710b72c5ad None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:15.670366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554fd01-6878-4f16-8c4f-21710b72c5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:15.670643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554fd01-6878-4f16-8c4f-21710b72c5ad None admin] PATH_INFO: `/v3/domains/de115f74853d4a96a40b33fc04c85607` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:15.671752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554fd01-6878-4f16-8c4f-21710b72c5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=de115f74853d4a96a40b33fc04c85607)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:15.673170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554fd01-6878-4f16-8c4f-21710b72c5ad None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:15.674804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 586/1176] 10.222.0.72 () {56 vars in 1183 bytes} [Tue Sep 13 12:48:15 2022] GET /identity/v3/domains/de115f74853d4a96a40b33fc04c85607 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:15.680599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17b425-1573-43e1-9d8c-f2b0be7dcad4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:15.775798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17b425-1573-43e1-9d8c-f2b0be7dcad4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:15.800373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:15.808522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf39384-fdbe-4b31-99e5-c5821ce3fb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:15.816785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39384-fdbe-4b31-99e5-c5821ce3fb87 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:15.816785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39384-fdbe-4b31-99e5-c5821ce3fb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:15.816785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39384-fdbe-4b31-99e5-c5821ce3fb87 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:15.816785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf39384-fdbe-4b31-99e5-c5821ce3fb87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:15.816785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf39384-fdbe-4b31-99e5-c5821ce3fb87 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:15.910404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 591/1177] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:48:15 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:48:18.078516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0d95-0493-4e44-98db-2c5b9ac871bd None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:18.079041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0d95-0493-4e44-98db-2c5b9ac871bd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:18.079041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e0d95-0493-4e44-98db-2c5b9ac871bd None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:18.080395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 587/1178] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:18.085637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf724e4-dc74-46bb-bd1d-6f6e12d37d7c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:18.086084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf724e4-dc74-46bb-bd1d-6f6e12d37d7c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:18.086212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf724e4-dc74-46bb-bd1d-6f6e12d37d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:18.103954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1bf724e4-dc74-46bb-bd1d-6f6e12d37d7c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:18.105138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1bf724e4-dc74-46bb-bd1d-6f6e12d37d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:18.168933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 592/1179] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:18.439539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0d95-0493-4e44-98db-2c5b9ac871bd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:18.489707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:18.538911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e0d95-0493-4e44-98db-2c5b9ac871bd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:18.571297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:18.571908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:18.572188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:18.572463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:18.583053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:48:18.587789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:48:18.588863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:48:18.590075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 588/1180] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:48:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:48:18.594993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf724e4-dc74-46bb-bd1d-6f6e12d37d7c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:18.691278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf724e4-dc74-46bb-bd1d-6f6e12d37d7c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:18.703844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:18.724796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:18.725396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:18.725672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:18.725971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:18.726624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:18.726923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:18.728285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:18.736319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 593/1181] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:18.742446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:18.839111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99085414-1473-4e09-a33d-3e9c11363aef None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:18.848647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:18.872814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:18.873434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:18.873712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:18.874010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] PATH_INFO: `/v3/users/89fa79a708d1492ab61e74509f071153` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:18.887553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] RBAC: Authorizing `identity:get_user(user_id=89fa79a708d1492ab61e74509f071153)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:18.888978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:18.890577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 589/1182] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:48:18 2022] GET /identity/v3/users/89fa79a708d1492ab61e74509f071153 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:18.896991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:18.999537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4634c21-1702-44d7-a7b3-ce33b3afcdf1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:19.032223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:19.033049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:19.033564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:19.033870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:19.074414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:19.098592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:19.100036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:19.193278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 594/1183] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:48:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:48:19.199423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:19.295059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320df73-1535-4a47-b7ef-41fadcc3c0fb None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:19.327700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e363-bb52-4f03-9aaa-d94aef38096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:19.328305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e363-bb52-4f03-9aaa-d94aef38096d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:19.328585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e363-bb52-4f03-9aaa-d94aef38096d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:19.328858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e363-bb52-4f03-9aaa-d94aef38096d None admin] PATH_INFO: `/v3/projects/3eba53b4695d4db5b7d3fae29488d441` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:19.330182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e363-bb52-4f03-9aaa-d94aef38096d None admin] RBAC: Authorizing `identity:get_project(project_id=3eba53b4695d4db5b7d3fae29488d441)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:19.331595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e363-bb52-4f03-9aaa-d94aef38096d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:19.333180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 590/1184] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:48:19 2022] GET /identity/v3/projects/3eba53b4695d4db5b7d3fae29488d441 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:19.339125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:19.410887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:19.434619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60247a01-c77e-4ed0-b304-43f175d89192 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:19.468112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384a024-2133-4cad-b039-59ef974c5a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:19.468774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384a024-2133-4cad-b039-59ef974c5a8f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:19.469055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384a024-2133-4cad-b039-59ef974c5a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:19.469319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384a024-2133-4cad-b039-59ef974c5a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:19.470113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384a024-2133-4cad-b039-59ef974c5a8f None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=3eba53b4695d4db5b7d3fae29488d441, user.id=89fa79a708d1492ab61e74509f071153) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:19.470457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384a024-2133-4cad-b039-59ef974c5a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:19.471880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384a024-2133-4cad-b039-59ef974c5a8f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:19.477143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 595/1185] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:48:19 2022] GET /identity/v3/role_assignments?user.id=89fa79a708d1492ab61e74509f071153&scope.project.id=3eba53b4695d4db5b7d3fae29488d441&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:21.590432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e38e9-16dd-4ddd-a1dc-6b1df236e3b7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:21.590962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e38e9-16dd-4ddd-a1dc-6b1df236e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:21.591070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e38e9-16dd-4ddd-a1dc-6b1df236e3b7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:21.592329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 591/1186] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:21.597777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c61d1f-7761-4b52-89c2-ef69d3d065d3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:21.598358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c61d1f-7761-4b52-89c2-ef69d3d065d3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:21.598358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c61d1f-7761-4b52-89c2-ef69d3d065d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:21.616434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-66c61d1f-7761-4b52-89c2-ef69d3d065d3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:21.617616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-66c61d1f-7761-4b52-89c2-ef69d3d065d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:21.682415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 596/1187] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:21.953501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e38e9-16dd-4ddd-a1dc-6b1df236e3b7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:21.988598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:22.052213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e38e9-16dd-4ddd-a1dc-6b1df236e3b7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:22.086888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:22.087511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:22.087789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:22.088059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:22.105084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:48:22.109797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:48:22.110975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:48:22.112452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 592/1188] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:48:22.119803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c61d1f-7761-4b52-89c2-ef69d3d065d3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:22.183666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:22.215486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c61d1f-7761-4b52-89c2-ef69d3d065d3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:22.248059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:22.248653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:22.248927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:22.249194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:22.249872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:22.250169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:22.251564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:22.259222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 597/1189] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:22.265008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:22.355543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:22.364214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1dd2-ee87-4e6c-85f6-7c167eb3ca2b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:22.398445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:22.399065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:22.399343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:22.399613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] PATH_INFO: `/v3/users/89fa79a708d1492ab61e74509f071153` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:22.400712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=89fa79a708d1492ab61e74509f071153)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:22.402147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:22.403822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 593/1190] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:48:22 2022] GET /identity/v3/users/89fa79a708d1492ab61e74509f071153 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:22.410423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:22.507833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81ea7-3649-46b9-8c3b-0f94d327f164 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:22.534388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:22.539929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4a777-1ee3-4d55-bd73-f8e3133cd720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:22.540552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a777-1ee3-4d55-bd73-f8e3133cd720 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:22.540830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a777-1ee3-4d55-bd73-f8e3133cd720 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:22.541101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a777-1ee3-4d55-bd73-f8e3133cd720 None admin] PATH_INFO: `/v3/projects/3eba53b4695d4db5b7d3fae29488d441` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:22.542222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4a777-1ee3-4d55-bd73-f8e3133cd720 None admin] RBAC: Authorizing `identity:get_project(project_id=3eba53b4695d4db5b7d3fae29488d441)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:22.543645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4a777-1ee3-4d55-bd73-f8e3133cd720 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:22.545251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 598/1191] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:48:22 2022] GET /identity/v3/projects/3eba53b4695d4db5b7d3fae29488d441 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:22.551929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:22.648515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6380ab-36f1-4d24-9ec3-c8d332db4ab1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:22.681350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054505c-8bb1-49f5-8427-5ee7d6521eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:22.682025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054505c-8bb1-49f5-8427-5ee7d6521eaa None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:22.682307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054505c-8bb1-49f5-8427-5ee7d6521eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:22.682582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054505c-8bb1-49f5-8427-5ee7d6521eaa None admin] PATH_INFO: `/v3/projects/3eba53b4695d4db5b7d3fae29488d441/users/89fa79a708d1492ab61e74509f071153/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:22.684442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054505c-8bb1-49f5-8427-5ee7d6521eaa None admin] RBAC: Authorizing `identity:create_grant(project_id=3eba53b4695d4db5b7d3fae29488d441, user_id=89fa79a708d1492ab61e74509f071153, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:22.685874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054505c-8bb1-49f5-8427-5ee7d6521eaa None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:22.749620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 594/1192] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:48:22 2022] PUT /identity/v3/projects/3eba53b4695d4db5b7d3fae29488d441/users/89fa79a708d1492ab61e74509f071153/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:48:24.868446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efe5f2-6ab1-44d8-8430-171794e5a74a None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:24.871940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efe5f2-6ab1-44d8-8430-171794e5a74a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:24.871940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efe5f2-6ab1-44d8-8430-171794e5a74a None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:24.871940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 599/1193] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:48:24.875537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea46f-ea5a-45b7-b02c-a1d96b005b81 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:24.875908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea46f-ea5a-45b7-b02c-a1d96b005b81 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:24.876072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea46f-ea5a-45b7-b02c-a1d96b005b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:24.894870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-372ea46f-ea5a-45b7-b02c-a1d96b005b81 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:48:24.895869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-372ea46f-ea5a-45b7-b02c-a1d96b005b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:48:24.912386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:24.962125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 595/1194] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:48:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:48:25.229514 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efe5f2-6ab1-44d8-8430-171794e5a74a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:25.328230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6efe5f2-6ab1-44d8-8430-171794e5a74a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:25.359599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:25.360212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:25.360489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:25.361277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:25.371449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:48:25.376168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:48:25.377801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:48:25.378766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 600/1195] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:48:25.383911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ea46f-ea5a-45b7-b02c-a1d96b005b81 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:25.478575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ea46f-ea5a-45b7-b02c-a1d96b005b81 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:25.511215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:25.511744 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:25.512020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:25.512289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:25.512938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:25.513235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:25.514603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:25.522240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 596/1196] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:25.528032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:25.584403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:25.626075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb891a49-2bb2-4610-8357-c60ff9b6ad45 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:25.660067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:25.660682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:25.660957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:25.661223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] PATH_INFO: `/v3/users/89fa79a708d1492ab61e74509f071153` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:25.662357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] RBAC: Authorizing `identity:get_user(user_id=89fa79a708d1492ab61e74509f071153)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:25.663865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:25.665412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 601/1197] 10.222.0.72 () {56 vars in 1177 bytes} [Tue Sep 13 12:48:25 2022] GET /identity/v3/users/89fa79a708d1492ab61e74509f071153 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:25.671214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:25.687946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:25.772253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0525d-7af9-469f-9c11-945f945dd00b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:25.806385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:25.806975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:25.807257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:25.807531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:25.873981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:25.875418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:25.889794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:25.973144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 597/1198] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:48:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:48:25.979218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:26.076560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d169fb4-7b82-42f8-84cd-f4001ffface1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:26.102367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:26.110614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7854cd-0c6e-42b4-a1a4-e9c0d08fa9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:26.114449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7854cd-0c6e-42b4-a1a4-e9c0d08fa9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:26.114449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7854cd-0c6e-42b4-a1a4-e9c0d08fa9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:26.114449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7854cd-0c6e-42b4-a1a4-e9c0d08fa9f0 None admin] PATH_INFO: `/v3/projects/3eba53b4695d4db5b7d3fae29488d441` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:26.114449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7854cd-0c6e-42b4-a1a4-e9c0d08fa9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=3eba53b4695d4db5b7d3fae29488d441)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:26.114813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7854cd-0c6e-42b4-a1a4-e9c0d08fa9f0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:26.116419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 602/1199] 10.222.0.72 () {56 vars in 1186 bytes} [Tue Sep 13 12:48:25 2022] GET /identity/v3/projects/3eba53b4695d4db5b7d3fae29488d441 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:26.122595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:48:26.218085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f385dab-da24-4b13-919e-35fcec96b07d None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:48:26.231495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:26.250403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ca904-bbd0-4fa6-9aac-d84af837cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:48:26.250993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca904-bbd0-4fa6-9aac-d84af837cf4a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:48:26.251277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca904-bbd0-4fa6-9aac-d84af837cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:48:26.251578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca904-bbd0-4fa6-9aac-d84af837cf4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:48:26.252441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ca904-bbd0-4fa6-9aac-d84af837cf4a None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=3eba53b4695d4db5b7d3fae29488d441, user.id=89fa79a708d1492ab61e74509f071153) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:48:26.252752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ca904-bbd0-4fa6-9aac-d84af837cf4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:48:26.254197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ca904-bbd0-4fa6-9aac-d84af837cf4a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:48:26.260716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 598/1200] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:48:26 2022] GET /identity/v3/role_assignments?user.id=89fa79a708d1492ab61e74509f071153&scope.project.id=3eba53b4695d4db5b7d3fae29488d441&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:48:36.132685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:48:36.259122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=11 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:39.567218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877cc9e-3eae-4983-b15f-d2b7bdf5aa46 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:39.567842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877cc9e-3eae-4983-b15f-d2b7bdf5aa46 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:39.567842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877cc9e-3eae-4983-b15f-d2b7bdf5aa46 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:39.569230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 603/1201] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:49:39.574790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7f0b8-a2d4-4abf-ab91-d60f389bc770 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:39.579199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7f0b8-a2d4-4abf-ab91-d60f389bc770 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:39.579199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7f0b8-a2d4-4abf-ab91-d60f389bc770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:39.593976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-92f7f0b8-a2d4-4abf-ab91-d60f389bc770 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:49:39.595168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-92f7f0b8-a2d4-4abf-ab91-d60f389bc770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:49:39.659966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 599/1202] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:49:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:49:39.932668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877cc9e-3eae-4983-b15f-d2b7bdf5aa46 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:40.038925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877cc9e-3eae-4983-b15f-d2b7bdf5aa46 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:40.073227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:40.073896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:40.074176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:40.074445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:40.085142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:40.086593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:40.101133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:40.102411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:40.103610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:40.103610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:40.103610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:40.103610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 604/1203] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:40.108600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7f0b8-a2d4-4abf-ab91-d60f389bc770 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:40.206649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7f0b8-a2d4-4abf-ab91-d60f389bc770 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:40.235141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:40.241616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51772e-0237-4c8c-adef-433f4925f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:40.242347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51772e-0237-4c8c-adef-433f4925f186 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:40.242632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51772e-0237-4c8c-adef-433f4925f186 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:40.242905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51772e-0237-4c8c-adef-433f4925f186 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:40.243541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51772e-0237-4c8c-adef-433f4925f186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:40.243853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51772e-0237-4c8c-adef-433f4925f186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:40.245394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51772e-0237-4c8c-adef-433f4925f186 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:40.257698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 600/1204] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:40.263699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:40.314864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:40.360214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c489fb2-94a8-4dd9-ad65-4b63098d49e2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:40.393533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef918b-86b9-4519-808d-78d958d37d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:40.394192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef918b-86b9-4519-808d-78d958d37d51 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:40.394472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef918b-86b9-4519-808d-78d958d37d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:40.394741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef918b-86b9-4519-808d-78d958d37d51 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:40.395394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef918b-86b9-4519-808d-78d958d37d51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:40.397436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef918b-86b9-4519-808d-78d958d37d51 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:40.527865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 605/1205] 10.222.0.72 () {60 vars in 1132 bytes} [Tue Sep 13 12:49:40 2022] POST /identity/v3/users => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:49:42.708585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab41a-039a-4491-a5e2-34ec19ed0274 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:42.709160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab41a-039a-4491-a5e2-34ec19ed0274 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:42.709160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab41a-039a-4491-a5e2-34ec19ed0274 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:42.711870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 601/1206] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:49:42.716254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f0363-4cd9-4650-8bf4-22595eca92d9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:42.716698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f0363-4cd9-4650-8bf4-22595eca92d9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:42.716806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f0363-4cd9-4650-8bf4-22595eca92d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:42.736191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-499f0363-4cd9-4650-8bf4-22595eca92d9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:49:42.737409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-499f0363-4cd9-4650-8bf4-22595eca92d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:49:42.809844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 606/1207] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:49:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:49:43.123179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab41a-039a-4491-a5e2-34ec19ed0274 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:43.254280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab41a-039a-4491-a5e2-34ec19ed0274 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:43.297130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:43.297784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:43.298063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:43.298334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:43.310177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:49:43.315133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:43.316873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:43.318116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 602/1208] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:43.323497 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f0363-4cd9-4650-8bf4-22595eca92d9 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:43.371398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:43.436658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f0363-4cd9-4650-8bf4-22595eca92d9 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:43.469694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:43.470326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:43.470605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:43.470877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:43.471533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:43.471833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:43.473196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:43.481725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 607/1209] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:43.487836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:43.536331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:43.584150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c91157-e648-4b95-8e33-3cac242ec41a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:43.617433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:43.618096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:43.618375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:43.618644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:43.628287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:43.629712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:43.644546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:43.645615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:43.646565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:43.646565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:43.646565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:43.646565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 603/1210] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:49:43.652078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:43.748696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:43.751544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de12e3c-81d1-4a36-8790-ec88da7a2b90 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:43.787267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:43.788514 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:43.788795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:43.789077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:43.789720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:43.790064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:43.791424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:43.804970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 608/1211] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:43.809439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:43.907872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372bb47-c017-4082-bd52-e7f70a8a918b None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:43.909383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:43.941729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:43.942425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:43.942712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:43.942981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:43.950828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:43.952257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:49:43.965360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:43.966541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:43.967541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 604/1212] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:43.972595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:44.073425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9377c0b-2984-4d19-a80d-0f6e055779e1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:44.107759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:44.108318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:44.108597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:44.108868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:44.109594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:44.109932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:44.111351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:44.127874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 609/1213] 10.222.0.72 () {56 vars in 1143 bytes} [Tue Sep 13 12:49:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:44.134435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:44.232748 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ac1db-7eb5-48af-a4f0-729759292261 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:44.266858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:44.267459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:44.267740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:44.268054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:44.297716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:44.335282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:44.336673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:44.435417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 605/1214] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:49:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:49:44.441577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:44.462455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:44.542035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7b193-990e-4e7a-be90-d686b0d31966 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:44.576605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:44.577218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:44.577492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:44.577782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:44.590439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:44.590439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:44.603284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:44.604421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:44.605530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:44.605530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:44.605530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:44.605530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 610/1215] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:44.610188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:44.671897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:44.714607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea6c86-f6f3-4696-a6cb-3c820ea87dc6 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:44.748373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:44.748717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:44.748997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:44.749272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:44.749943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:44.750246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:44.751602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:44.763073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 606/1216] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:44.768813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:44.836785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:44.867166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d160ae5-a0ea-4180-8b6d-00fb29c9553f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:44.899733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:44.900349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:44.900629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:44.900968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:44.908337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:44.909841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:49:44.921395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:44.922523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:44.923506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 611/1217] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:44.929092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:45.027946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c8e6-33ac-4f7a-a53c-62dbef2cdebc None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:45.050564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:45.062996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa9036-8753-4765-adcd-ad44d379ad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:45.063602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa9036-8753-4765-adcd-ad44d379ad7e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:45.063905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa9036-8753-4765-adcd-ad44d379ad7e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:45.064185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa9036-8753-4765-adcd-ad44d379ad7e None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:45.064876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa9036-8753-4765-adcd-ad44d379ad7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:45.065185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa9036-8753-4765-adcd-ad44d379ad7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:45.066632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa9036-8753-4765-adcd-ad44d379ad7e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:45.079242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 607/1218] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:45.085607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:45.184160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16704ad-088a-4e37-bc1c-f97c240aa419 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:45.212550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:45.218174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7d03a-0d20-4513-8ead-d9d378c33a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:45.218768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7d03a-0d20-4513-8ead-d9d378c33a26 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:45.219046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7d03a-0d20-4513-8ead-d9d378c33a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:45.219315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7d03a-0d20-4513-8ead-d9d378c33a26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:45.220030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7d03a-0d20-4513-8ead-d9d378c33a26 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=dca4df9896df47268d091c3b32cbe8d6) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:45.220338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7d03a-0d20-4513-8ead-d9d378c33a26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:45.221761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7d03a-0d20-4513-8ead-d9d378c33a26 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:45.227864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 612/1219] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:49:45 2022] GET /identity/v3/role_assignments?user.id=dca4df9896df47268d091c3b32cbe8d6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:47.339926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb834-949d-4d7b-88fe-9cea6c17effe None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:47.340480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb834-949d-4d7b-88fe-9cea6c17effe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:47.340480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb834-949d-4d7b-88fe-9cea6c17effe None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:47.341872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 608/1220] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:49:47.346900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92cfab-ee2a-40f2-8cac-7f9999be306c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:47.347296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92cfab-ee2a-40f2-8cac-7f9999be306c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:47.350635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92cfab-ee2a-40f2-8cac-7f9999be306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:47.366112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2a92cfab-ee2a-40f2-8cac-7f9999be306c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:49:47.367308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2a92cfab-ee2a-40f2-8cac-7f9999be306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:49:47.431923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 613/1221] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:49:47.701930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fb834-949d-4d7b-88fe-9cea6c17effe None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:47.801926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fb834-949d-4d7b-88fe-9cea6c17effe None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:47.834848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:47.835466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:47.835745 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:47.836018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:47.846299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:49:47.853486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:47.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:47.856836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 609/1222] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:49:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:47.858932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92cfab-ee2a-40f2-8cac-7f9999be306c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:47.955904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92cfab-ee2a-40f2-8cac-7f9999be306c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:47.978962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:47.989191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:47.989815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:47.990092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:47.990358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:47.991622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:47.991952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:47.993302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:48.000867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 614/1223] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:49:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:49:48.007341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:48.051560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:48.109627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95ed6c-d52a-4f5e-985c-d45edb3452e3 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:48.142921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:48.143544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:48.143827 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:48.144101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:48.155404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:48.160581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:48.171786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:48.172942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:48.174048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:48.174048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:48.174048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:48.174048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 610/1224] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:49:48.179399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:48.280591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880f215-19bb-4fee-8387-eb4872f685b7 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:48.314068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:48.314662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:48.314942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:48.315213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:48.315889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:48.316191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:48.317551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:48.329753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 615/1225] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:49:48.336067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:48.444460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:48.451411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09b9ee-c8b6-4975-9684-6adc41d25081 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:48.484265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:48.484923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:48.485199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:48.485473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:48.492497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:48.494393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:49:48.505707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:48.507359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:48.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 611/1226] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:48.513682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:48.558878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:48.619307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0be3c-d34e-42db-8ebf-890c92e67e7b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:48.654217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:48.654814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:48.655089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:48.655354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:48.656065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:48.656392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:48.657830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:48.674557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 616/1227] 10.222.0.72 () {56 vars in 1143 bytes} [Tue Sep 13 12:49:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:48.681191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:48.783003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751817f-b80f-4b83-8b48-ccda29f708c2 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:48.816791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:48.817373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:48.817657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:48.817961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:48.829604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:48.831082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:48.845431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:48.846445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:48.847649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:48.847649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:48.847649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:48.847649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 612/1228] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:48.852979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:48.953051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5e944-60ca-4e5a-94de-f247d6ec7de1 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:48.955334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:48.985931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:48.986532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:48.988782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:48.988782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:48.988782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:48.988782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:48.989330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:49.000759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 617/1229] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:49.007683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:49.017996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:49.104824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d9a20-d634-4158-9df9-baadec01cf71 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:49.137504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:49.138237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:49.138513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:49.138781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:49.145231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:49.146813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:49:49.157850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:49.158942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:49.160052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 613/1230] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:49.165015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:49.263230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26fcb1-65d3-47d4-99e7-43cb163623a2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:49.296555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfcf62-60c8-4c7f-bebf-4e8ce9cbdb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:49.297158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfcf62-60c8-4c7f-bebf-4e8ce9cbdb89 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:49.297441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfcf62-60c8-4c7f-bebf-4e8ce9cbdb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:49.297714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfcf62-60c8-4c7f-bebf-4e8ce9cbdb89 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:49.298431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfcf62-60c8-4c7f-bebf-4e8ce9cbdb89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:49.298737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfcf62-60c8-4c7f-bebf-4e8ce9cbdb89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:49.300154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfcf62-60c8-4c7f-bebf-4e8ce9cbdb89 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:49.312477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 618/1231] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:49.318704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:49.383266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:49.417907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3658ef2-1c67-4ac8-85a8-975cf39a0b44 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:49.450514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad3ba1-c87c-4c30-a716-14b7ba585be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:49.451165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad3ba1-c87c-4c30-a716-14b7ba585be8 None admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:49.451441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad3ba1-c87c-4c30-a716-14b7ba585be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:49.451716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad3ba1-c87c-4c30-a716-14b7ba585be8 None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/dca4df9896df47268d091c3b32cbe8d6/roles/fd66e13437394d4f98a74978756fb6e0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:49.466006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad3ba1-c87c-4c30-a716-14b7ba585be8 None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=dca4df9896df47268d091c3b32cbe8d6, role_id=fd66e13437394d4f98a74978756fb6e0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:49.467419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad3ba1-c87c-4c30-a716-14b7ba585be8 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:49.576068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 614/1232] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:49:49 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/dca4df9896df47268d091c3b32cbe8d6/roles/fd66e13437394d4f98a74978756fb6e0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:49:51.725954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c239384-9f88-44f9-82f2-17498c5e23bf None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:51.726537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c239384-9f88-44f9-82f2-17498c5e23bf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:51.726537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c239384-9f88-44f9-82f2-17498c5e23bf None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:51.727865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 619/1233] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:49:51.733044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6774-6787-42ce-a9c0-4ea8164ffdfc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:51.733477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6774-6787-42ce-a9c0-4ea8164ffdfc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:51.733601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6774-6787-42ce-a9c0-4ea8164ffdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:51.751388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-265a6774-6787-42ce-a9c0-4ea8164ffdfc None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:49:51.752608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-265a6774-6787-42ce-a9c0-4ea8164ffdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:49:51.817590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 615/1234] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:49:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:49:52.089836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c239384-9f88-44f9-82f2-17498c5e23bf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:52.114384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:52.187823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c239384-9f88-44f9-82f2-17498c5e23bf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:52.221821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:52.222480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:52.222750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:52.223016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:52.233217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:49:52.238006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:52.239094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:52.240223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 620/1235] 10.222.0.72 () {56 vars in 1102 bytes} [Tue Sep 13 12:49:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:52.245946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6774-6787-42ce-a9c0-4ea8164ffdfc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:52.292585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:52.342899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6774-6787-42ce-a9c0-4ea8164ffdfc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:52.375808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:52.376405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:52.376689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:52.377022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:52.377679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] RBAC: Adding query filter params (name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:52.378010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:52.379360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:52.386995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 616/1236] 10.222.0.72 () {56 vars in 1103 bytes} [Tue Sep 13 12:49:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:52.393055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:52.476338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:52.491620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2271285-76ae-431f-a7ff-0a082d2ab385 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:52.524415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:52.525028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:52.525303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:52.525570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:52.535667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:52.537086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:52.550618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:52.551781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:52.553602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:52.553602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:52.553602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:52.553602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 621/1237] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:52.558210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:52.655128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623aebb-8cc7-4ffd-a74f-df57f255d089 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:52.663759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:52.687986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:52.688626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:52.688859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:52.689126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:52.689783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:52.690084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:52.691437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:52.704529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 617/1238] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:52.710523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:52.807552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3220f01-1154-4a0d-9ea8-9a687273d0d2 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:52.840633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:52.841249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:52.841525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:52.841820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:52.844408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:52.849955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:52.851611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:49:52.862474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:52.863593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:52.864786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:52.869041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 622/1239] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:52.870265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:52.971476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2cf7f-0d4c-4aef-b135-39d62446a8cd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:53.007358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:53.007957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:53.008242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:53.008520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:53.009247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:53.009557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:53.011129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:53.028246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 618/1240] 10.222.0.72 () {56 vars in 1143 bytes} [Tue Sep 13 12:49:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:53.034095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:53.133287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6471-6aeb-4d9c-9720-6d223fd351ae None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:53.166301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:53.166902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:53.167178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:53.167446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:53.226422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:53.232710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:53.237228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:53.329449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 623/1241] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:49:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:49:53.335869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:53.362891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:53.433045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187b862-e66d-4d00-934d-33e7dd2cb193 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:53.468924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:53.469548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:53.469859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:53.470136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:53.481400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:53.482812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:53.497893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:53.499008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:53.500102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:53.500102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:53.500102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:53.500102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 619/1242] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:49:53.504781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:53.597911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:53.604353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bc192-a004-485b-8766-77bf7125d861 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:53.637463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:53.638102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:53.638379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:53.638649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:53.639453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:53.639751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:53.641105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:53.653094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 624/1243] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:53.658863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:53.737434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:53.758417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1104ebb-9b88-4255-a42a-bdb191a12d20 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:53.790973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:53.791596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:53.791871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:53.792141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:53.798872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:53.800557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:49:53.811901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:53.817369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:53.818999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 620/1244] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:53.820216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:53.918364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19960abd-4ffc-4114-9745-3cd2ba6ce88e None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:53.952117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642381c3-cc9c-41f6-8ce4-18a0eda7e317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:53.952724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642381c3-cc9c-41f6-8ce4-18a0eda7e317 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:53.953003 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642381c3-cc9c-41f6-8ce4-18a0eda7e317 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:53.953277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642381c3-cc9c-41f6-8ce4-18a0eda7e317 None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:53.953993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642381c3-cc9c-41f6-8ce4-18a0eda7e317 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:53.954297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642381c3-cc9c-41f6-8ce4-18a0eda7e317 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:53.955714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642381c3-cc9c-41f6-8ce4-18a0eda7e317 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:53.968011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 625/1245] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:53.974194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:54.072890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18dedf-9109-4156-8bf8-576adae75b5f None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:54.105537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0089fe-ab1e-4059-84b6-2986d733fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:54.106138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0089fe-ab1e-4059-84b6-2986d733fc72 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:54.106423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0089fe-ab1e-4059-84b6-2986d733fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:54.106701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0089fe-ab1e-4059-84b6-2986d733fc72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:54.107431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0089fe-ab1e-4059-84b6-2986d733fc72 None admin] RBAC: Adding query filter params (role.id=fd66e13437394d4f98a74978756fb6e0, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=dca4df9896df47268d091c3b32cbe8d6) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:54.107741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0089fe-ab1e-4059-84b6-2986d733fc72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:54.109159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0089fe-ab1e-4059-84b6-2986d733fc72 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:54.115769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 621/1246] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:49:53 2022] GET /identity/v3/role_assignments?user.id=dca4df9896df47268d091c3b32cbe8d6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=fd66e13437394d4f98a74978756fb6e0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:56.328359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3435461-02da-43db-a7ac-77d115411569 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:56.328880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3435461-02da-43db-a7ac-77d115411569 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:56.328880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3435461-02da-43db-a7ac-77d115411569 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:56.330283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 626/1247] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:49:56.335792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc393cc-b278-4cca-9927-89296e78106a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:56.336197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc393cc-b278-4cca-9927-89296e78106a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:56.336356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc393cc-b278-4cca-9927-89296e78106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:56.342680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:56.355476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ecc393cc-b278-4cca-9927-89296e78106a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:49:56.356664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ecc393cc-b278-4cca-9927-89296e78106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:49:56.423030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 622/1248] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:49:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:49:56.695354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3435461-02da-43db-a7ac-77d115411569 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:56.713389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:56.796245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3435461-02da-43db-a7ac-77d115411569 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:56.830272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:56.830892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:56.831169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:56.831436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:56.841299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:49:56.846160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:56.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:56.848458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 627/1249] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 12:49:56.853937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc393cc-b278-4cca-9927-89296e78106a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:56.949174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc393cc-b278-4cca-9927-89296e78106a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:56.978102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:56.982808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:56.983410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:56.983690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:56.984087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:56.984750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:56.985056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:56.986435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:56.995040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 623/1250] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:57.001281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:57.078819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:57.102249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1bdba-3228-46c1-bee8-798c6eab3eb4 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:57.134907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:57.135535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:57.135819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:57.136095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:57.147645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:57.149131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:57.163520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:57.164667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:57.165779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:57.165779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:57.165779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:57.165779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 628/1251] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:57.170464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:57.268765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f633fa-cae5-4448-993e-ec978980d087 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:57.302420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:57.303096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:57.303385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:57.303657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:57.304335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:57.304638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:57.306021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:57.317820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 624/1252] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:57.323956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:57.422233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806883fd-0e9f-4202-a2db-ceaac7f79ecc None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:57.448254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:57.456172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:57.456793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:57.457072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:57.457399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:57.465596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:57.467073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:49:57.478044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:57.479202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:49:57.480468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 629/1253] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:57.485052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:57.539525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:57.582800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc19c5-51d4-4e42-9475-e6973a9d61c1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:57.615863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:57.616460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:57.616741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:57.617014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:57.617759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:57.618069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:57.619468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:57.636453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 625/1254] 10.222.0.72 () {56 vars in 1143 bytes} [Tue Sep 13 12:49:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:57.642229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:57.740307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2368b05-2ebb-4de4-8c32-5f023c60407f None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:57.773883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:57.774480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:57.774757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:57.775025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:57.829624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:57.841051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:57.842520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:57.936681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 630/1255] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:49:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:49:57.942620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:58.041392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16f3ef-a1ab-4e1a-af4f-082608a46ed0 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:58.054550 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:58.075310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:58.076089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:58.076241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:58.076516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:58.086511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:58.087891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:49:58.103003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:58.104070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:49:58.104954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:49:58.104954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:49:58.104954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:58.104954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 626/1256] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:58.110323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:58.197078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:58.208446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577408-6839-4695-9adc-a8d74df99ebe None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:58.240373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:58.241470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:58.241753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:58.242037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:58.242667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:58.242968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:58.244305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:58.255051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 631/1257] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:58.261419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:58.359942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e7b1b-ae93-4cd1-adac-18ca7174d523 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:58.393294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:58.393945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:58.394227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:58.394495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:58.401236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:58.406088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:49:58.413768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:49:58.414798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:49:58.415929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 627/1258] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:49:58.420684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:58.518042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71597508-b012-4eb4-9cf6-4ede3f731789 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:58.550028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4390a42-d335-48a7-96ee-0a8c80747fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:58.550620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4390a42-d335-48a7-96ee-0a8c80747fed None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:58.550895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4390a42-d335-48a7-96ee-0a8c80747fed None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:58.551163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4390a42-d335-48a7-96ee-0a8c80747fed None admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:58.551888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4390a42-d335-48a7-96ee-0a8c80747fed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:58.552187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4390a42-d335-48a7-96ee-0a8c80747fed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:58.554177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4390a42-d335-48a7-96ee-0a8c80747fed None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:58.566008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 632/1259] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:49:58.572816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:49:58.598024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:49:58.671060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29f934-ab03-4e3d-add9-e94d29627c72 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:49:58.704449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19726e4-191d-49b6-a650-988fb18c274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:49:58.705042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19726e4-191d-49b6-a650-988fb18c274c None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:49:58.705320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19726e4-191d-49b6-a650-988fb18c274c None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:49:58.705589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19726e4-191d-49b6-a650-988fb18c274c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:49:58.706338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19726e4-191d-49b6-a650-988fb18c274c None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=dca4df9896df47268d091c3b32cbe8d6) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:49:58.706643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19726e4-191d-49b6-a650-988fb18c274c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:49:58.708107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19726e4-191d-49b6-a650-988fb18c274c None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:49:58.713290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 628/1260] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:49:58 2022] GET /identity/v3/role_assignments?user.id=dca4df9896df47268d091c3b32cbe8d6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:50:00.837215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1974be-d829-4f38-bdd9-6df0cef432aa None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:00.837789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1974be-d829-4f38-bdd9-6df0cef432aa None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:00.837789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1974be-d829-4f38-bdd9-6df0cef432aa None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:00.839172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 633/1261] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:50:00.844277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333245-4cfd-466f-a9fc-0285ea7a9e41 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:00.844683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333245-4cfd-466f-a9fc-0285ea7a9e41 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:00.844831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333245-4cfd-466f-a9fc-0285ea7a9e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:00.863110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ee333245-4cfd-466f-a9fc-0285ea7a9e41 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:50:00.864292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ee333245-4cfd-466f-a9fc-0285ea7a9e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:50:00.930998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 629/1262] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:50:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:50:01.200841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1974be-d829-4f38-bdd9-6df0cef432aa None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:01.212091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:01.299503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1974be-d829-4f38-bdd9-6df0cef432aa None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:01.334345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:01.335627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:01.335903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:01.336177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:01.346154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:50:01.351574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:01.352694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:50:01.353800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 634/1263] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:01.358719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333245-4cfd-466f-a9fc-0285ea7a9e41 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:01.376467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:01.457127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333245-4cfd-466f-a9fc-0285ea7a9e41 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:01.491635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:01.495198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:01.495198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:01.495198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:01.495198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:01.495198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:01.495198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:01.504560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 630/1264] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:01.510538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:01.576359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:01.609036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95617b68-9a4b-42f5-ae50-a2926c0e62b5 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:01.641268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:01.641932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:01.642215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:01.642487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:01.653688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:01.655150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:50:01.668929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:01.670217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:50:01.671622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:50:01.671622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:01.671622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:50:01.671622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 635/1265] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:01.675856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:01.757955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:01.776199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b23cda-ffc4-4073-9bbe-62ef12a14e5a None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:01.808107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:01.808709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:01.808992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:01.810313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:01.810313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:01.810313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:01.811609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:01.823705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 631/1266] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:01.829461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:01.929811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1f24-3e46-4510-8967-af6fd0c1fb45 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:01.948903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:01.963191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:01.963807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:01.964093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:01.964348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:01.972074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:01.973490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:50:01.984829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:01.986006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:50:01.987168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 636/1267] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:50:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:01.992381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:02.090752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901feba9-31dd-4d77-ae49-f6deb6dfe878 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:02.126047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:02.126683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:02.127056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:02.127335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:02.128111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:02.128444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:02.129888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:02.139022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:02.148069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 632/1268] 10.222.0.72 () {56 vars in 1143 bytes} [Tue Sep 13 12:50:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:02.153885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:02.255529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f61c-8334-471a-b331-cdb578c4cedc None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:02.288330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:02.289182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:02.289459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:02.289730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:02.299785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:02.301243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:50:02.315343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:02.316542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:50:02.317586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:50:02.317586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:02.317586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:50:02.317586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 637/1269] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:02.323620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:02.423531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bf29a-eb8b-472f-8c79-7da1a996b772 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:02.457036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:02.457625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:02.457932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:02.458202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:02.458836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:02.459133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:02.460511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:02.471299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 633/1270] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:02.478925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:02.510531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:02.576340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d465-6b16-42bd-ba6c-798e22b1bf91 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:02.612194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:02.612813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:02.613092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:02.613363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:02.620701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:02.622163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:50:02.633586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:02.634712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:50:02.636329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 638/1271] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:02.640632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:02.698882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:02.740052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5febf-23d1-4d23-b562-efd4c92763b7 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:02.773356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ec6f7-c9ee-4bdf-9e13-588b92ce74bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:02.774009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ec6f7-c9ee-4bdf-9e13-588b92ce74bc None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:02.774296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ec6f7-c9ee-4bdf-9e13-588b92ce74bc None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:02.774575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ec6f7-c9ee-4bdf-9e13-588b92ce74bc None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:02.775390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ec6f7-c9ee-4bdf-9e13-588b92ce74bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:02.775730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ec6f7-c9ee-4bdf-9e13-588b92ce74bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:02.777266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ec6f7-c9ee-4bdf-9e13-588b92ce74bc None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:02.790396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 634/1272] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:02.796400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:02.885709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:02.894235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ef2a4-6be6-46a6-9d71-34cdb4a55900 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:02.927581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ef3ba-8af5-469f-8e43-6a7fae63eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:02.928272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef3ba-8af5-469f-8e43-6a7fae63eb9b None admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:02.928684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef3ba-8af5-469f-8e43-6a7fae63eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:02.929064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef3ba-8af5-469f-8e43-6a7fae63eb9b None admin] PATH_INFO: `/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/dca4df9896df47268d091c3b32cbe8d6/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:02.930932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ef3ba-8af5-469f-8e43-6a7fae63eb9b None admin] RBAC: Authorizing `identity:create_grant(project_id=03397d33d3b1442597a329d1ae8b7543, user_id=dca4df9896df47268d091c3b32cbe8d6, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:02.932385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ef3ba-8af5-469f-8e43-6a7fae63eb9b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:03.027926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 639/1273] 10.222.0.72 () {58 vars in 1439 bytes} [Tue Sep 13 12:50:02 2022] PUT /identity/v3/projects/03397d33d3b1442597a329d1ae8b7543/users/dca4df9896df47268d091c3b32cbe8d6/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 12:50:05.171787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e316d0-721f-4327-9823-a558667f52d8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:05.172375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e316d0-721f-4327-9823-a558667f52d8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:05.172375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e316d0-721f-4327-9823-a558667f52d8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:05.173670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 635/1274] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:50:05.178927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254f94-23ed-4a17-ad67-fc12ac854afc None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:05.179322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254f94-23ed-4a17-ad67-fc12ac854afc None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:05.179473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28254f94-23ed-4a17-ad67-fc12ac854afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:05.197719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-28254f94-23ed-4a17-ad67-fc12ac854afc None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:50:05.198916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-28254f94-23ed-4a17-ad67-fc12ac854afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:50:05.266169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 640/1275] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:50:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:50:05.541605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e316d0-721f-4327-9823-a558667f52d8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:05.641918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e316d0-721f-4327-9823-a558667f52d8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:05.660652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:05.677175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:05.677822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:05.678796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:05.679153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:05.688778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 12:50:05.693501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:05.694714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:05.695760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 636/1276] 10.222.0.72 () {56 vars in 1096 bytes} [Tue Sep 13 12:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:05.700906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28254f94-23ed-4a17-ad67-fc12ac854afc None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:05.781026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:05.799287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28254f94-23ed-4a17-ad67-fc12ac854afc None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:05.832288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:05.832886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:05.833158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:05.833425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:05.834122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:05.834900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:05.836259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:05.843883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 641/1277] 10.222.0.72 () {56 vars in 1099 bytes} [Tue Sep 13 12:50:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:05.849682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:05.948316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11a04b-3a1b-4eb2-a71a-b9ef756f4713 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:05.981968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:05.982583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:05.982859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:05.983125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:05.992937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:05.994415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:50:06.008277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:06.009337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:50:06.010341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:50:06.010341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:06.010341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:06.010341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 637/1278] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:06.015135 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:06.111884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3893bc-7cf4-4421-91ac-a64337d845c9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:06.144284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:06.144876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:06.145152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:06.145479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:06.146139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:06.146444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:06.148386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:06.159773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 642/1279] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:06.166147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:06.196887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:06.264282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a1d7f-6191-4521-8b25-d85156bb4379 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:06.297872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:06.298492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:06.298798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:06.299074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:06.306398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:06.308076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 12:50:06.320047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:06.322084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:06.323910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 638/1280] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:50:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:06.327332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:06.331464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:06.427078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b0ab2-a554-4ca8-88d3-4e7d4a7630c0 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:06.460649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:06.461243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:06.461517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:06.461813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:06.462545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:06.462840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:06.464252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:06.480775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 643/1281] 10.222.0.72 () {56 vars in 1143 bytes} [Tue Sep 13 12:50:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:06.486695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:06.574625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:06.583845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c959d63-c188-40f2-99cd-7bb1eb42ce07 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:06.616829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:06.617435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:06.617715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:06.618008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:06.684466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:06.685848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:06.773125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:06.783054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 639/1282] 10.222.0.72 () {58 vars in 1282 bytes} [Tue Sep 13 12:50:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 12:50:06.789144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:06.850464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:06.887253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b793-38d0-4cba-9e89-d115388370da None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:06.925178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:06.925840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:06.926121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:06.926394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:06.937677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:06.939093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 12:50:06.954641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:06.955740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 12:50:06.956992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 12:50:06.956992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 12:50:06.956992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:50:06.956992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 644/1283] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:06.962572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:07.059340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8249ca2-cdb3-42bb-907d-6e3e382ad936 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:07.091803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:07.092391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:07.092670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:07.092941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:07.093571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:07.093896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:07.095280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:07.106711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 640/1284] 10.222.0.72 () {56 vars in 1109 bytes} [Tue Sep 13 12:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 12:50:07.112332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:07.214483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e5ce5-09cf-46b3-9d14-1bbfd04eaec9 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:07.228599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:07.247531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:07.248144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:07.248440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:07.248689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:07.254985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:07.256431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 12:50:07.268097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:07.273283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: ERROR keystone.server.flask.application Sep 13 12:50:07.274458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 645/1285] 10.222.0.72 () {56 vars in 1111 bytes} [Tue Sep 13 12:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:07.275764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:07.325510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:07.373749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1179c-3470-48ab-a6f3-4e9589dec5a1 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:07.405112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e76142-d217-46c0-b0ee-06db13db6049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:07.405707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e76142-d217-46c0-b0ee-06db13db6049 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:07.406014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e76142-d217-46c0-b0ee-06db13db6049 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:07.406285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e76142-d217-46c0-b0ee-06db13db6049 None admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:07.406965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e76142-d217-46c0-b0ee-06db13db6049 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:07.407265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e76142-d217-46c0-b0ee-06db13db6049 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:07.408664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e76142-d217-46c0-b0ee-06db13db6049 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:07.422460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 641/1286] 10.222.0.72 () {56 vars in 1148 bytes} [Tue Sep 13 12:50:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:07.428742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:07.532131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bfb7c-5331-4326-ae4c-9ed6181cd051 None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:07.565035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a61a0e-8d3c-421b-b6b7-b0ea53a555d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:07.565631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61a0e-8d3c-421b-b6b7-b0ea53a555d4 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:07.569163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61a0e-8d3c-421b-b6b7-b0ea53a555d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:07.569163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a61a0e-8d3c-421b-b6b7-b0ea53a555d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:07.569163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a61a0e-8d3c-421b-b6b7-b0ea53a555d4 None admin] RBAC: Adding query filter params (role.id=0c2f57ac71af4bb6be296c0ff7db64de, scope.project.id=03397d33d3b1442597a329d1ae8b7543, user.id=dca4df9896df47268d091c3b32cbe8d6) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:07.569163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a61a0e-8d3c-421b-b6b7-b0ea53a555d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:07.569163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a61a0e-8d3c-421b-b6b7-b0ea53a555d4 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:07.574890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 646/1287] 10.222.0.72 () {56 vars in 1374 bytes} [Tue Sep 13 12:50:07 2022] GET /identity/v3/role_assignments?user.id=dca4df9896df47268d091c3b32cbe8d6&scope.project.id=03397d33d3b1442597a329d1ae8b7543&role.id=0c2f57ac71af4bb6be296c0ff7db64de => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:09.769365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3ea66-27f5-4a36-97ea-f18adabccc42 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:09.769951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3ea66-27f5-4a36-97ea-f18adabccc42 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:09.769951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3ea66-27f5-4a36-97ea-f18adabccc42 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:09.771350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 642/1288] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:50:09.776970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17653d-2f59-45cb-8e32-93809fdb074f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:09.778822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17653d-2f59-45cb-8e32-93809fdb074f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:09.778822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17653d-2f59-45cb-8e32-93809fdb074f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:09.795948 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7b17653d-2f59-45cb-8e32-93809fdb074f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:50:09.797172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7b17653d-2f59-45cb-8e32-93809fdb074f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:50:09.827227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:09.863439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 647/1289] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:50:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:50:10.132067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3ea66-27f5-4a36-97ea-f18adabccc42 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:10.233038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3ea66-27f5-4a36-97ea-f18adabccc42 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:10.243425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:10.265055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:10.265667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:10.265975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:10.266248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:10.266825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:10.268221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:10.277107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:10.278414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:10.279473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 643/1290] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:50:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:10.284267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17653d-2f59-45cb-8e32-93809fdb074f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:10.383277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17653d-2f59-45cb-8e32-93809fdb074f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:10.421464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e6b04-72a2-4ced-939a-7a30a237b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:10.422110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6b04-72a2-4ced-939a-7a30a237b710 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:10.422385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6b04-72a2-4ced-939a-7a30a237b710 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:10.422655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6b04-72a2-4ced-939a-7a30a237b710 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:10.423276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e6b04-72a2-4ced-939a-7a30a237b710 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:10.423574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e6b04-72a2-4ced-939a-7a30a237b710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:10.424926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e6b04-72a2-4ced-939a-7a30a237b710 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:10.429913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 648/1291] 10.222.0.72 () {56 vars in 1116 bytes} [Tue Sep 13 12:50:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:10.435707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:10.535009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d7a58-ca0d-4e4c-833a-9fd3f686c778 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:10.567351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ec47b-c962-4c30-842e-55b47034ea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:10.567944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec47b-c962-4c30-842e-55b47034ea0f None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:10.568219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec47b-c962-4c30-842e-55b47034ea0f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:10.568487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ec47b-c962-4c30-842e-55b47034ea0f None admin] PATH_INFO: `/v3/services` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:10.569714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ec47b-c962-4c30-842e-55b47034ea0f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:10.570043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ec47b-c962-4c30-842e-55b47034ea0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:10.571667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ec47b-c962-4c30-842e-55b47034ea0f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:10.575935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 644/1292] 10.222.0.72 () {56 vars in 1116 bytes} [Tue Sep 13 12:50:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:12.704638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1031d6-faea-4007-a3ab-7bd684e18442 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:12.705165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1031d6-faea-4007-a3ab-7bd684e18442 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:12.705165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1031d6-faea-4007-a3ab-7bd684e18442 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:12.706611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 649/1293] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:50:12.712281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be50b-da51-4e4a-a600-7e507eaa140d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:12.712723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be50b-da51-4e4a-a600-7e507eaa140d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:12.712897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be50b-da51-4e4a-a600-7e507eaa140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:12.731980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b61be50b-da51-4e4a-a600-7e507eaa140d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:50:12.733170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b61be50b-da51-4e4a-a600-7e507eaa140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:50:12.762897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:12.801180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 645/1294] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:50:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:50:13.070439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1031d6-faea-4007-a3ab-7bd684e18442 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:13.101581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:13.171569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1031d6-faea-4007-a3ab-7bd684e18442 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:13.204792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32104c-ad72-4722-8c83-c64f461f4421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:13.205386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32104c-ad72-4722-8c83-c64f461f4421 None admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:13.205663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32104c-ad72-4722-8c83-c64f461f4421 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:13.205960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32104c-ad72-4722-8c83-c64f461f4421 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:13.206600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32104c-ad72-4722-8c83-c64f461f4421 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:13.207950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32104c-ad72-4722-8c83-c64f461f4421 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:13.301275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 650/1295] 10.222.0.72 () {60 vars in 1141 bytes} [Tue Sep 13 12:50:13 2022] POST /identity/v3/services => generated 246 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 12:50:15.441410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc8f4e-9a56-43dc-99bb-7f7eea4bb17c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:15.441973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc8f4e-9a56-43dc-99bb-7f7eea4bb17c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:15.442175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc8f4e-9a56-43dc-99bb-7f7eea4bb17c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:15.443350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 646/1296] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:50:15.448989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13236-0c2d-44a4-8546-4ab569b4f857 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:15.449424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13236-0c2d-44a4-8546-4ab569b4f857 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:15.449554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13236-0c2d-44a4-8546-4ab569b4f857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:15.468168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d4e13236-0c2d-44a4-8546-4ab569b4f857 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:50:15.469351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d4e13236-0c2d-44a4-8546-4ab569b4f857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:50:15.542322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 651/1297] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:50:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 12:50:15.811062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc8f4e-9a56-43dc-99bb-7f7eea4bb17c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:15.911211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc8f4e-9a56-43dc-99bb-7f7eea4bb17c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:15.946714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:15.947325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:15.947698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:15.947972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:15.948542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:15.949906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:15.958443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:15.959661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:15.960513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 647/1298] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:50:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:15.965824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13236-0c2d-44a4-8546-4ab569b4f857 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:15.984508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:16.062495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13236-0c2d-44a4-8546-4ab569b4f857 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:16.095802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:16.096399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:16.096699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:16.096973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:16.097597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:16.097929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:16.099293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:16.104949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 652/1299] 10.222.0.72 () {56 vars in 1116 bytes} [Tue Sep 13 12:50:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:16.110912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:16.142635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:16.208037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a52228-5c1d-4235-abce-fb6fa2dcc926 None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:16.240880 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18474ecc-a981-43f9-88c1-ddcee28d866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:16.241473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18474ecc-a981-43f9-88c1-ddcee28d866e None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:16.241779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18474ecc-a981-43f9-88c1-ddcee28d866e None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:16.242055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18474ecc-a981-43f9-88c1-ddcee28d866e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:16.242773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18474ecc-a981-43f9-88c1-ddcee28d866e None admin] RBAC: Adding query filter params (interface=public, service_id=ed5af92df435436288d7f2f8f8009236, region_id=RegionOne) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:16.243095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18474ecc-a981-43f9-88c1-ddcee28d866e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:16.245410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18474ecc-a981-43f9-88c1-ddcee28d866e None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:16.250330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 648/1300] 10.222.0.72 () {56 vars in 1251 bytes} [Tue Sep 13 12:50:16 2022] GET /identity/v3/endpoints?service_id=ed5af92df435436288d7f2f8f8009236&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:16.255904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:16.334786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:16.352393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05df57-35a9-459e-92e0-4daf9a2e9d1b None admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:16.385144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df980f-cb4f-4d35-a3f4-518966aca470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:16.385752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df980f-cb4f-4d35-a3f4-518966aca470 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:16.386039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df980f-cb4f-4d35-a3f4-518966aca470 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:16.386310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df980f-cb4f-4d35-a3f4-518966aca470 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:16.386904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df980f-cb4f-4d35-a3f4-518966aca470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:16.388263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df980f-cb4f-4d35-a3f4-518966aca470 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:16.394811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 653/1301] 10.222.0.72 () {56 vars in 1087 bytes} [Tue Sep 13 12:50:16 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 12:50:18.557312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51960e77-33cc-439e-8d6d-83dc815108ae None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:18.557969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51960e77-33cc-439e-8d6d-83dc815108ae None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:18.557969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51960e77-33cc-439e-8d6d-83dc815108ae None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:18.559263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 649/1302] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:50:18.564672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbdb32-2533-434a-934c-da73dc78ff1d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:18.565100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbdb32-2533-434a-934c-da73dc78ff1d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:18.565216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbdb32-2533-434a-934c-da73dc78ff1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:18.583402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-8cfbdb32-2533-434a-934c-da73dc78ff1d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:50:18.585114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8cfbdb32-2533-434a-934c-da73dc78ff1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:50:18.651735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 654/1303] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:50:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 12:50:18.926255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51960e77-33cc-439e-8d6d-83dc815108ae None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:19.019540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:19.026857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51960e77-33cc-439e-8d6d-83dc815108ae None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:19.059444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:19.060086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:19.060365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:19.060638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:19.061214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:19.062627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 12:50:19.072108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 12:50:19.073288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application with Lock( Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self._enter() Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application return self.creator() Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application created_value = creator( Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: ERROR keystone.server.flask.application Sep 13 12:50:19.074234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 650/1304] 10.222.0.72 () {56 vars in 1117 bytes} [Tue Sep 13 12:50:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 12:50:19.079238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbdb32-2533-434a-934c-da73dc78ff1d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:19.157831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:19.180234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbdb32-2533-434a-934c-da73dc78ff1d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:19.216033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127aec7-76d9-4431-bb6c-ac02f9fdcb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:19.216627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127aec7-76d9-4431-bb6c-ac02f9fdcb69 None admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:19.216906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127aec7-76d9-4431-bb6c-ac02f9fdcb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:19.217180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127aec7-76d9-4431-bb6c-ac02f9fdcb69 None admin] PATH_INFO: `/v3/services` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:19.217826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127aec7-76d9-4431-bb6c-ac02f9fdcb69 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 12:50:19.218129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127aec7-76d9-4431-bb6c-ac02f9fdcb69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:19.219492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127aec7-76d9-4431-bb6c-ac02f9fdcb69 None admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:19.224400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 655/1305] 10.222.0.72 () {56 vars in 1116 bytes} [Tue Sep 13 12:50:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 12:50:19.230325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:50:19.330875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361ee84-83a5-4cf7-a675-4d0f41907bfd None admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:50:19.363210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a358cb-9237-4bc7-ab2b-ae7a7ed2246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:50:19.363799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a358cb-9237-4bc7-ab2b-ae7a7ed2246f None admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:50:19.364078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a358cb-9237-4bc7-ab2b-ae7a7ed2246f None admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:50:19.364350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a358cb-9237-4bc7-ab2b-ae7a7ed2246f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:50:19.365016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a358cb-9237-4bc7-ab2b-ae7a7ed2246f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:50:19.367069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a358cb-9237-4bc7-ab2b-ae7a7ed2246f None admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:50:19.468902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 651/1306] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 12:50:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 12:50:29.221446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=30 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:50:29.464382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:36.602634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045838-831f-4f06-a496-76badbc35ee6 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:36.603145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045838-831f-4f06-a496-76badbc35ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:36.603145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56045838-831f-4f06-a496-76badbc35ee6 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:36.604831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 656/1307] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:36.610143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b62e8-31af-407d-8332-53e7a49eb73d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:36.610581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b62e8-31af-407d-8332-53e7a49eb73d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:36.610695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b62e8-31af-407d-8332-53e7a49eb73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:36.763454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1b4b62e8-31af-407d-8332-53e7a49eb73d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:36.765053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1b4b62e8-31af-407d-8332-53e7a49eb73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:36.887918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 652/1308] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:37.187176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f316-e37a-4261-a3fb-4a3626cd0441 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:37.187453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f316-e37a-4261-a3fb-4a3626cd0441 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:37.188069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f316-e37a-4261-a3fb-4a3626cd0441 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:37.189690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 657/1309] 10.222.0.72 () {54 vars in 965 bytes} [Tue Sep 13 12:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:37.196062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ab0e7-f667-4221-b3b3-747e1698ef83 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:37.196594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ab0e7-f667-4221-b3b3-747e1698ef83 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:37.196877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ab0e7-f667-4221-b3b3-747e1698ef83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:37.292404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-536ab0e7-f667-4221-b3b3-747e1698ef83 None None] MFA Rules not processed for user `4ed50a15319e4991ae90b0e5271f4c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:37.293763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-536ab0e7-f667-4221-b3b3-747e1698ef83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:37.316708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:37.376854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 653/1310] 10.222.0.72 () {58 vars in 1062 bytes} [Tue Sep 13 12:56:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:37.386659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308f316-e37a-4261-a3fb-4a3626cd0441 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:56:37.388455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4308f316-e37a-4261-a3fb-4a3626cd0441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:37.510381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308f316-e37a-4261-a3fb-4a3626cd0441 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:56:37.591147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb597b70-db6c-4edd-bfcc-4d59862b88e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:56:37.591888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb597b70-db6c-4edd-bfcc-4d59862b88e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:37.592171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb597b70-db6c-4edd-bfcc-4d59862b88e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:37.592446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb597b70-db6c-4edd-bfcc-4d59862b88e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:37.594176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cb597b70-db6c-4edd-bfcc-4d59862b88e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:37.674589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:37.732085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb597b70-db6c-4edd-bfcc-4d59862b88e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:56:37.734283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb597b70-db6c-4edd-bfcc-4d59862b88e4 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:56:37.753693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 658/1311] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:56:40.220619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b06440-bd1d-46a0-b65b-33c0b7ea741e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:40.221144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b06440-bd1d-46a0-b65b-33c0b7ea741e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:40.221144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b06440-bd1d-46a0-b65b-33c0b7ea741e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:40.222579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 654/1312] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:40.227924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f8808-d4c5-400a-a5a6-c61ae030a0d7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:40.228359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f8808-d4c5-400a-a5a6-c61ae030a0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:40.228454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f8808-d4c5-400a-a5a6-c61ae030a0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:40.304152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-347f8808-d4c5-400a-a5a6-c61ae030a0d7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:40.305534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-347f8808-d4c5-400a-a5a6-c61ae030a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:40.402127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 659/1313] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:40.686660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b06440-bd1d-46a0-b65b-33c0b7ea741e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:56:40.688295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c9b06440-bd1d-46a0-b65b-33c0b7ea741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:40.799722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b06440-bd1d-46a0-b65b-33c0b7ea741e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:56:40.874663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:40.876936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034e7a4-89dc-4402-965b-3b665014daf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:56:40.877547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e7a4-89dc-4402-965b-3b665014daf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:40.877860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e7a4-89dc-4402-965b-3b665014daf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:40.878135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034e7a4-89dc-4402-965b-3b665014daf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:40.879723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3034e7a4-89dc-4402-965b-3b665014daf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:40.989924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e7a4-89dc-4402-965b-3b665014daf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:56:40.991374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034e7a4-89dc-4402-965b-3b665014daf4 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:56:41.009383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 655/1314] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:56:43.706075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424da1-2f91-45db-aba1-0c222bfda621 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:43.706651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424da1-2f91-45db-aba1-0c222bfda621 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:43.706651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424da1-2f91-45db-aba1-0c222bfda621 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:43.707960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 660/1315] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:43.713231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ad8f-d7a8-4b0d-9c14-a42d494e8a99 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:43.713684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ad8f-d7a8-4b0d-9c14-a42d494e8a99 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:43.713812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ad8f-d7a8-4b0d-9c14-a42d494e8a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:43.778954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-49a8ad8f-d7a8-4b0d-9c14-a42d494e8a99 None None] MFA Rules not processed for user `d332b1b14aa24ef8b6ae8d8929faf9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:43.780137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-49a8ad8f-d7a8-4b0d-9c14-a42d494e8a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:43.830673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:43.875060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 656/1316] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:56:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:56:44.159700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99424da1-2f91-45db-aba1-0c222bfda621 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:56:44.161224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-99424da1-2f91-45db-aba1-0c222bfda621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:44.171593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:44.263005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99424da1-2f91-45db-aba1-0c222bfda621 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:56:44.334400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b930c03-be72-4310-a442-c287a7b89140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:56:44.334997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b930c03-be72-4310-a442-c287a7b89140 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:44.335267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b930c03-be72-4310-a442-c287a7b89140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:44.335566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b930c03-be72-4310-a442-c287a7b89140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:44.337103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6b930c03-be72-4310-a442-c287a7b89140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:44.398748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:44.444226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b930c03-be72-4310-a442-c287a7b89140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:56:44.445574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b930c03-be72-4310-a442-c287a7b89140 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:56:44.463817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 661/1317] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:56:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:56:47.428760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83a1fa-e842-4ea0-bb8a-245bc543a2cb None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:47.429298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83a1fa-e842-4ea0-bb8a-245bc543a2cb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:47.429298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83a1fa-e842-4ea0-bb8a-245bc543a2cb None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:47.430707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 657/1318] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:47.436836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ce42d-511f-4862-a05f-7dca1f7ecc38 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:47.437252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ce42d-511f-4862-a05f-7dca1f7ecc38 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:47.437365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ce42d-511f-4862-a05f-7dca1f7ecc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:47.504062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-965ce42d-511f-4862-a05f-7dca1f7ecc38 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:47.505294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-965ce42d-511f-4862-a05f-7dca1f7ecc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:47.597435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 662/1319] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:56:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:47.881161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83a1fa-e842-4ea0-bb8a-245bc543a2cb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:56:47.882731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ca83a1fa-e842-4ea0-bb8a-245bc543a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:47.984409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83a1fa-e842-4ea0-bb8a-245bc543a2cb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:56:48.057231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a89005-69f1-4027-9c82-90c734c81c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:56:48.057852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a89005-69f1-4027-9c82-90c734c81c1f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:48.058129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a89005-69f1-4027-9c82-90c734c81c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:48.058826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a89005-69f1-4027-9c82-90c734c81c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:48.060414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-38a89005-69f1-4027-9c82-90c734c81c1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:48.067090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:48.169831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a89005-69f1-4027-9c82-90c734c81c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:56:48.171298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a89005-69f1-4027-9c82-90c734c81c1f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:56:48.189438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 658/1320] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:56:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:56:51.374978 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75e3d-51e4-4105-ad8c-bba027723537 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:51.375564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75e3d-51e4-4105-ad8c-bba027723537 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:51.375564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75e3d-51e4-4105-ad8c-bba027723537 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:51.376961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 663/1321] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:56:51.382180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4025-5c8d-4488-aca1-7705f1f2473e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:51.387662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4025-5c8d-4488-aca1-7705f1f2473e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:51.387662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4025-5c8d-4488-aca1-7705f1f2473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:51.451791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-aaaf4025-5c8d-4488-aca1-7705f1f2473e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:51.453015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-aaaf4025-5c8d-4488-aca1-7705f1f2473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:51.510317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:51.549177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 659/1322] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:51.836611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff75e3d-51e4-4105-ad8c-bba027723537 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:56:51.838381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-aff75e3d-51e4-4105-ad8c-bba027723537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:51.862911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:51.953255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff75e3d-51e4-4105-ad8c-bba027723537 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:56:52.033122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b5b5b-1d7d-416a-a539-d52a2dc9931c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:56:52.033628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b5b5b-1d7d-416a-a539-d52a2dc9931c service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:52.033938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b5b5b-1d7d-416a-a539-d52a2dc9931c service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:52.034209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b5b5b-1d7d-416a-a539-d52a2dc9931c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:52.035822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d81b5b5b-1d7d-416a-a539-d52a2dc9931c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:52.112982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:52.151296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b5b5b-1d7d-416a-a539-d52a2dc9931c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:56:52.153710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b5b5b-1d7d-416a-a539-d52a2dc9931c service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:56:52.173503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 664/1323] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:56:54.410950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846cd33-1018-42d3-ba4d-d6c08e047577 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:54.411558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846cd33-1018-42d3-ba4d-d6c08e047577 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:54.411558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846cd33-1018-42d3-ba4d-d6c08e047577 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:54.412950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 660/1324] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:54.419637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764857-c7f3-44df-8d27-cf657856ab38 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:54.420086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764857-c7f3-44df-8d27-cf657856ab38 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:54.420349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764857-c7f3-44df-8d27-cf657856ab38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:54.489526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-18764857-c7f3-44df-8d27-cf657856ab38 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:54.490778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-18764857-c7f3-44df-8d27-cf657856ab38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:54.591348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 665/1325] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:54.874452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846cd33-1018-42d3-ba4d-d6c08e047577 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:56:54.875988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c846cd33-1018-42d3-ba4d-d6c08e047577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:54.980951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846cd33-1018-42d3-ba4d-d6c08e047577 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:56:55.054295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6984b20-8646-4d4e-abaa-8ea3bc6f7a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:56:55.054820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6984b20-8646-4d4e-abaa-8ea3bc6f7a8f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:55.055096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6984b20-8646-4d4e-abaa-8ea3bc6f7a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:55.055364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6984b20-8646-4d4e-abaa-8ea3bc6f7a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:55.056940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c6984b20-8646-4d4e-abaa-8ea3bc6f7a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:55.075310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:55.166339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6984b20-8646-4d4e-abaa-8ea3bc6f7a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:56:55.168278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6984b20-8646-4d4e-abaa-8ea3bc6f7a8f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:56:55.186770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 661/1326] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:56:57.397307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3658b6-0b1c-4331-b84a-eb38720683d8 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:57.397842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3658b6-0b1c-4331-b84a-eb38720683d8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:57.398102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3658b6-0b1c-4331-b84a-eb38720683d8 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:57.400942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 666/1327] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:56:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:57.405831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0c0e8-d350-4f8c-babd-513af4b0427a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:57.406259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0c0e8-d350-4f8c-babd-513af4b0427a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:57.406351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0c0e8-d350-4f8c-babd-513af4b0427a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:57.482043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9ca0c0e8-d350-4f8c-babd-513af4b0427a None None] MFA Rules not processed for user `d332b1b14aa24ef8b6ae8d8929faf9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:57.483177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9ca0c0e8-d350-4f8c-babd-513af4b0427a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:57.548631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:57.580085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 662/1328] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:57.886872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838f8ef-e47c-4f14-ae32-7d5f28ca89ea None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:57.887338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838f8ef-e47c-4f14-ae32-7d5f28ca89ea None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:57.887664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838f8ef-e47c-4f14-ae32-7d5f28ca89ea None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:57.889048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 667/1329] 10.222.0.72 () {54 vars in 965 bytes} [Tue Sep 13 12:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:56:57.895168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d6fed-512d-46f2-a392-6f523dadcf66 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:57.895461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d6fed-512d-46f2-a392-6f523dadcf66 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:57.895737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d6fed-512d-46f2-a392-6f523dadcf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:57.976071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d26d6fed-512d-46f2-a392-6f523dadcf66 None None] MFA Rules not processed for user `4ed50a15319e4991ae90b0e5271f4c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:56:57.977973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d26d6fed-512d-46f2-a392-6f523dadcf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:58.058959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 663/1330] 10.222.0.72 () {58 vars in 1062 bytes} [Tue Sep 13 12:56:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:56:58.068125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838f8ef-e47c-4f14-ae32-7d5f28ca89ea None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:56:58.069660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5838f8ef-e47c-4f14-ae32-7d5f28ca89ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:58.104739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:58.180810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838f8ef-e47c-4f14-ae32-7d5f28ca89ea None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:56:58.268726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a2d20-68c4-45ca-9147-57ea0bb8dafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:56:58.269452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a2d20-68c4-45ca-9147-57ea0bb8dafa service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:56:58.270238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a2d20-68c4-45ca-9147-57ea0bb8dafa service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:56:58.270238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a2d20-68c4-45ca-9147-57ea0bb8dafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:56:58.271703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c77a2d20-68c4-45ca-9147-57ea0bb8dafa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:56:58.356968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:56:58.389133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a2d20-68c4-45ca-9147-57ea0bb8dafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:56:58.390606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a2d20-68c4-45ca-9147-57ea0bb8dafa service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:56:58.409157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 668/1331] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:56:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:01.439448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222afc90-d42f-4863-b6ad-54efa775be41 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:01.439986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222afc90-d42f-4863-b6ad-54efa775be41 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:01.439986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222afc90-d42f-4863-b6ad-54efa775be41 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:01.441346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 664/1332] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:01.446693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45951ec-2564-4f07-af6c-0adbc7bc85f0 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:01.447099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45951ec-2564-4f07-af6c-0adbc7bc85f0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:01.447243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45951ec-2564-4f07-af6c-0adbc7bc85f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:01.512124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-c45951ec-2564-4f07-af6c-0adbc7bc85f0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:01.513321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c45951ec-2564-4f07-af6c-0adbc7bc85f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:01.606568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 669/1333] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:01.887846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222afc90-d42f-4863-b6ad-54efa775be41 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:01.889430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-222afc90-d42f-4863-b6ad-54efa775be41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:01.960634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:01.994167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222afc90-d42f-4863-b6ad-54efa775be41 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:02.070341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a3456-541a-4891-ab81-9f33cef1a514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:02.070948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a3456-541a-4891-ab81-9f33cef1a514 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:02.071232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a3456-541a-4891-ab81-9f33cef1a514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:02.071507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a3456-541a-4891-ab81-9f33cef1a514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:02.073105 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f23a3456-541a-4891-ab81-9f33cef1a514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:02.185495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a3456-541a-4891-ab81-9f33cef1a514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:02.186895 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a3456-541a-4891-ab81-9f33cef1a514 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:02.196573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:02.204707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 665/1334] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:04.383692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff34e5-538f-49c1-831f-9ce06146cd21 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:04.384231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff34e5-538f-49c1-831f-9ce06146cd21 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:04.384231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff34e5-538f-49c1-831f-9ce06146cd21 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:04.385561 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 670/1335] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:04.390899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b29c1-1232-43b2-898d-9c7e3df4fbc9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:04.391322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b29c1-1232-43b2-898d-9c7e3df4fbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:04.391459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b29c1-1232-43b2-898d-9c7e3df4fbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:04.456856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-326b29c1-1232-43b2-898d-9c7e3df4fbc9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:04.458098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-326b29c1-1232-43b2-898d-9c7e3df4fbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:04.552909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 666/1336] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:04.841834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff34e5-538f-49c1-831f-9ce06146cd21 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:04.843384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3eff34e5-538f-49c1-831f-9ce06146cd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:04.883044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:04.946952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff34e5-538f-49c1-831f-9ce06146cd21 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:05.019048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98111fc6-37f1-48e8-9916-02997de360c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:05.019676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98111fc6-37f1-48e8-9916-02997de360c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:05.019929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98111fc6-37f1-48e8-9916-02997de360c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:05.020202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98111fc6-37f1-48e8-9916-02997de360c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:05.021805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-98111fc6-37f1-48e8-9916-02997de360c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:05.105679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:05.130966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98111fc6-37f1-48e8-9916-02997de360c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:05.132395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98111fc6-37f1-48e8-9916-02997de360c6 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:05.151342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 671/1337] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:08.038228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea61f0-429d-4f92-a784-9c317a7a6b5a None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:08.038782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea61f0-429d-4f92-a784-9c317a7a6b5a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:08.038782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea61f0-429d-4f92-a784-9c317a7a6b5a None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:08.040120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 667/1338] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:08.045694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4bcf0-12b8-4728-a264-e8ef0c4dcef9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:08.046104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4bcf0-12b8-4728-a264-e8ef0c4dcef9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:08.046271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4bcf0-12b8-4728-a264-e8ef0c4dcef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:08.111107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-31d4bcf0-12b8-4728-a264-e8ef0c4dcef9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:08.112306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-31d4bcf0-12b8-4728-a264-e8ef0c4dcef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:08.207756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 672/1339] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:08.496046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea61f0-429d-4f92-a784-9c317a7a6b5a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:08.497682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0dea61f0-429d-4f92-a784-9c317a7a6b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:08.572783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:08.599919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea61f0-429d-4f92-a784-9c317a7a6b5a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:08.677255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a966-5b09-4156-9193-d434e18df0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:08.677898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a966-5b09-4156-9193-d434e18df0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:08.678182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a966-5b09-4156-9193-d434e18df0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:08.678452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a966-5b09-4156-9193-d434e18df0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:08.680065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b2a5a966-5b09-4156-9193-d434e18df0b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:08.786370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a966-5b09-4156-9193-d434e18df0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:08.787784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a966-5b09-4156-9193-d434e18df0b3 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:08.805867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 668/1340] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:13.237861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ccee2-2488-46e8-924e-458c66b8c2b3 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:13.238384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ccee2-2488-46e8-924e-458c66b8c2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:13.238384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ccee2-2488-46e8-924e-458c66b8c2b3 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:13.239756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 673/1341] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:13.245763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c29f7-d975-4fa3-b2c8-fc8e54fac1bc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:13.246341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c29f7-d975-4fa3-b2c8-fc8e54fac1bc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:13.246540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c29f7-d975-4fa3-b2c8-fc8e54fac1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:13.252123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:13.315615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d52c29f7-d975-4fa3-b2c8-fc8e54fac1bc None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:13.316829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d52c29f7-d975-4fa3-b2c8-fc8e54fac1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:13.415813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 669/1342] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:57:13.703371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ccee2-2488-46e8-924e-458c66b8c2b3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:13.705026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c08ccee2-2488-46e8-924e-458c66b8c2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:13.755937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:13.811632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ccee2-2488-46e8-924e-458c66b8c2b3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:13.886102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b2a34-8d0a-4ad0-93c7-31e0d40cdd2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:13.886898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b2a34-8d0a-4ad0-93c7-31e0d40cdd2c service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:13.887173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b2a34-8d0a-4ad0-93c7-31e0d40cdd2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:13.887439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b2a34-8d0a-4ad0-93c7-31e0d40cdd2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:13.889056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a01b2a34-8d0a-4ad0-93c7-31e0d40cdd2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:13.989952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:14.006035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b2a34-8d0a-4ad0-93c7-31e0d40cdd2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:14.012987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b2a34-8d0a-4ad0-93c7-31e0d40cdd2c service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:14.025715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 674/1343] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:17.173866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8786-879b-445f-a2e9-aff0f4de7173 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:17.174441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8786-879b-445f-a2e9-aff0f4de7173 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:17.174441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b8786-879b-445f-a2e9-aff0f4de7173 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:17.175772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 670/1344] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:17.181084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc4808-6f0f-469b-9514-cd7fdb90b900 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:17.181531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc4808-6f0f-469b-9514-cd7fdb90b900 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:17.181624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc4808-6f0f-469b-9514-cd7fdb90b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:17.253975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-26cc4808-6f0f-469b-9514-cd7fdb90b900 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:17.261207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-26cc4808-6f0f-469b-9514-cd7fdb90b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:17.354995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 675/1345] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:17.642444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8786-879b-445f-a2e9-aff0f4de7173 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:17.644022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a62b8786-879b-445f-a2e9-aff0f4de7173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:17.732198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:17.749063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b8786-879b-445f-a2e9-aff0f4de7173 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:17.825956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf59cb-cf03-41b7-a1f9-3cd01a80ec8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:17.826564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf59cb-cf03-41b7-a1f9-3cd01a80ec8c service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:17.826839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf59cb-cf03-41b7-a1f9-3cd01a80ec8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:17.827106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf59cb-cf03-41b7-a1f9-3cd01a80ec8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:17.828697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ecf59cb-cf03-41b7-a1f9-3cd01a80ec8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:17.948145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf59cb-cf03-41b7-a1f9-3cd01a80ec8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:17.949991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf59cb-cf03-41b7-a1f9-3cd01a80ec8c service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:17.973775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 671/1346] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:22.288890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f11a5-3148-48c9-bf31-c219b6b83f81 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:22.289403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f11a5-3148-48c9-bf31-c219b6b83f81 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:22.289530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f11a5-3148-48c9-bf31-c219b6b83f81 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:22.290978 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 676/1347] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:22.296607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275255a-f571-4cc7-bc47-31702342d715 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:22.297085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275255a-f571-4cc7-bc47-31702342d715 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:22.297202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275255a-f571-4cc7-bc47-31702342d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:22.316190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:22.373905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6275255a-f571-4cc7-bc47-31702342d715 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:22.375123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6275255a-f571-4cc7-bc47-31702342d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:22.477205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 672/1348] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:57:22.767233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f11a5-3148-48c9-bf31-c219b6b83f81 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:22.768766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-559f11a5-3148-48c9-bf31-c219b6b83f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:22.827539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:22.872794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f11a5-3148-48c9-bf31-c219b6b83f81 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:22.944797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ac94e-415d-4b49-8c35-21761cd8a04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:22.945402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac94e-415d-4b49-8c35-21761cd8a04a service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:22.945678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac94e-415d-4b49-8c35-21761cd8a04a service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:22.946236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ac94e-415d-4b49-8c35-21761cd8a04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:22.947846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-366ac94e-415d-4b49-8c35-21761cd8a04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:23.053261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:23.056921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ac94e-415d-4b49-8c35-21761cd8a04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:23.059221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ac94e-415d-4b49-8c35-21761cd8a04a service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:23.077796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 677/1349] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:28.158003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e237-072a-4e1f-bd9c-8d96f89c93e3 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:28.158449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e237-072a-4e1f-bd9c-8d96f89c93e3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:28.158833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e237-072a-4e1f-bd9c-8d96f89c93e3 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:28.160309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 673/1350] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 12:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:28.166086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b6b8a-ae4c-4659-a83c-a8846f772903 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:28.166919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b6b8a-ae4c-4659-a83c-a8846f772903 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:28.167388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b6b8a-ae4c-4659-a83c-a8846f772903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:28.270694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-858b6b8a-ae4c-4659-a83c-a8846f772903 None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:28.272256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-858b6b8a-ae4c-4659-a83c-a8846f772903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:28.454188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 678/1351] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 12:57:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:32.475020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=59 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:32.590500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8e5fb-156f-4278-a46a-0dc2b75fa821 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:32.591019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8e5fb-156f-4278-a46a-0dc2b75fa821 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:32.591442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8e5fb-156f-4278-a46a-0dc2b75fa821 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:32.592929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 674/1352] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 12:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:32.598221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d6d98-d20f-420a-b7c8-378470111189 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:32.599086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d6d98-d20f-420a-b7c8-378470111189 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:32.599577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d6d98-d20f-420a-b7c8-378470111189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:32.665066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:32.690464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-012d6d98-d20f-420a-b7c8-378470111189 None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:32.692491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-012d6d98-d20f-420a-b7c8-378470111189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:32.819291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 679/1353] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 12:57:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:34.154799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fd623-964c-4384-8a11-5ffe9ca6ac6b None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:34.155302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fd623-964c-4384-8a11-5ffe9ca6ac6b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:34.155302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fd623-964c-4384-8a11-5ffe9ca6ac6b None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:34.156600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 675/1354] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 12:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:34.161721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf30bae-8a58-442a-ba8b-7ca11dcac7e5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:34.162719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf30bae-8a58-442a-ba8b-7ca11dcac7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:34.163060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf30bae-8a58-442a-ba8b-7ca11dcac7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:34.247316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-bcf30bae-8a58-442a-ba8b-7ca11dcac7e5 None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:34.248524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bcf30bae-8a58-442a-ba8b-7ca11dcac7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:34.291993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:34.358198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 680/1355] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 12:57:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:38.106749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e69dc-eb84-484d-bb04-80c72bf59653 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:38.107294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e69dc-eb84-484d-bb04-80c72bf59653 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:38.107294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e69dc-eb84-484d-bb04-80c72bf59653 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:38.108618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 676/1356] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:38.114566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c5ca3-34f8-4bc4-af48-a3493989e7ad None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:38.115010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c5ca3-34f8-4bc4-af48-a3493989e7ad None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:38.115108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c5ca3-34f8-4bc4-af48-a3493989e7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:38.190202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1c7c5ca3-34f8-4bc4-af48-a3493989e7ad None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:38.191305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1c7c5ca3-34f8-4bc4-af48-a3493989e7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:38.301509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 681/1357] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:57:38.608064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cf54-7f1d-4e4c-ad22-1912e46d6de9 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:38.608982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cf54-7f1d-4e4c-ad22-1912e46d6de9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:38.609513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94cf54-7f1d-4e4c-ad22-1912e46d6de9 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:38.611861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 677/1358] 10.222.0.72 () {54 vars in 962 bytes} [Tue Sep 13 12:57:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:38.620007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6d67b-1a8f-41e9-be5a-95e55da78f18 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:38.620292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6d67b-1a8f-41e9-be5a-95e55da78f18 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:38.620559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6d67b-1a8f-41e9-be5a-95e55da78f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:38.638084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:38.707381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2bc6d67b-1a8f-41e9-be5a-95e55da78f18 None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:38.709114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2bc6d67b-1a8f-41e9-be5a-95e55da78f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:38.827965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 682/1359] 10.222.0.72 () {58 vars in 1059 bytes} [Tue Sep 13 12:57:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:38.835538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94cf54-7f1d-4e4c-ad22-1912e46d6de9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:38.837149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7f94cf54-7f1d-4e4c-ad22-1912e46d6de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:39.001767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94cf54-7f1d-4e4c-ad22-1912e46d6de9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:39.081436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:39.124529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9af942-6b84-4c20-8f8e-0e34e8724fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:39.125215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9af942-6b84-4c20-8f8e-0e34e8724fee service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:39.125440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9af942-6b84-4c20-8f8e-0e34e8724fee service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:39.125670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9af942-6b84-4c20-8f8e-0e34e8724fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:39.127423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7c9af942-6b84-4c20-8f8e-0e34e8724fee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:39.227610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9af942-6b84-4c20-8f8e-0e34e8724fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:39.229922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9af942-6b84-4c20-8f8e-0e34e8724fee service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:39.250045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 678/1360] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:41.667658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a64bdf-741c-481c-8649-b10d8a2955a6 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:41.668096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a64bdf-741c-481c-8649-b10d8a2955a6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:41.668589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a64bdf-741c-481c-8649-b10d8a2955a6 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:41.670465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 683/1361] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:41.675233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf795-e668-4b80-8c4a-8322358270cc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:41.675233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf795-e668-4b80-8c4a-8322358270cc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:41.676012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdf795-e668-4b80-8c4a-8322358270cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:41.744840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f4cdf795-e668-4b80-8c4a-8322358270cc None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:41.746068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f4cdf795-e668-4b80-8c4a-8322358270cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:41.785650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:41.851329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 679/1362] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:42.148849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a64bdf-741c-481c-8649-b10d8a2955a6 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:42.151214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-46a64bdf-741c-481c-8649-b10d8a2955a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:42.231104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:42.341290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a64bdf-741c-481c-8649-b10d8a2955a6 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:42.477582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849df73-db4e-4d00-8703-62b35e3eef03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:42.478220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849df73-db4e-4d00-8703-62b35e3eef03 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:42.479162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849df73-db4e-4d00-8703-62b35e3eef03 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:42.479436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849df73-db4e-4d00-8703-62b35e3eef03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:42.481047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d849df73-db4e-4d00-8703-62b35e3eef03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:42.503954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:42.574396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849df73-db4e-4d00-8703-62b35e3eef03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:42.576157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849df73-db4e-4d00-8703-62b35e3eef03 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:42.595759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 684/1363] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:45.267721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805b683-0dda-4243-818e-22ecf385b4d7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:45.268334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805b683-0dda-4243-818e-22ecf385b4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:45.268334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805b683-0dda-4243-818e-22ecf385b4d7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:45.269763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 680/1364] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:45.274899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3b54-72c3-4472-93e1-579c6e5edff2 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:45.275487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3b54-72c3-4472-93e1-579c6e5edff2 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:45.275576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3b54-72c3-4472-93e1-579c6e5edff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:45.346470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9a5a3b54-72c3-4472-93e1-579c6e5edff2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:45.347677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9a5a3b54-72c3-4472-93e1-579c6e5edff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:45.451542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:45.462117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 685/1365] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:45.748064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659596d1-4c0e-4b49-a866-0147541ce642 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:45.755373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659596d1-4c0e-4b49-a866-0147541ce642 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:45.755373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659596d1-4c0e-4b49-a866-0147541ce642 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:45.755373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 681/1366] 10.222.0.72 () {54 vars in 962 bytes} [Tue Sep 13 12:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:45.756051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252e6fe-0b8e-46ae-b44d-22d11c9b491f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:45.756331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252e6fe-0b8e-46ae-b44d-22d11c9b491f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:45.756598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252e6fe-0b8e-46ae-b44d-22d11c9b491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:45.835809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2252e6fe-0b8e-46ae-b44d-22d11c9b491f None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:45.837082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2252e6fe-0b8e-46ae-b44d-22d11c9b491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:45.959788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 686/1367] 10.222.0.72 () {58 vars in 1059 bytes} [Tue Sep 13 12:57:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:45.968389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659596d1-4c0e-4b49-a866-0147541ce642 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:45.970119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-659596d1-4c0e-4b49-a866-0147541ce642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:46.124294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659596d1-4c0e-4b49-a866-0147541ce642 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:46.145792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:46.243425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18930f0c-60a7-4813-8758-6cd4757abdb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:46.244639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18930f0c-60a7-4813-8758-6cd4757abdb9 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:46.245011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18930f0c-60a7-4813-8758-6cd4757abdb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:46.245448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18930f0c-60a7-4813-8758-6cd4757abdb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:46.247142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-18930f0c-60a7-4813-8758-6cd4757abdb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:46.346200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18930f0c-60a7-4813-8758-6cd4757abdb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:46.347934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18930f0c-60a7-4813-8758-6cd4757abdb9 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:46.366710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 682/1368] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:46.406171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb32e-8a2f-4491-9b7b-a5dc3430becb None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:46.406722 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb32e-8a2f-4491-9b7b-a5dc3430becb None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:46.406891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb32e-8a2f-4491-9b7b-a5dc3430becb None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:46.408367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 687/1369] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:46.413797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdb214-48ac-435e-acd6-d83d3452e65b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:46.414092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdb214-48ac-435e-acd6-d83d3452e65b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:46.414365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdb214-48ac-435e-acd6-d83d3452e65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:46.447035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:46.484386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-13fdb214-48ac-435e-acd6-d83d3452e65b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:46.486113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-13fdb214-48ac-435e-acd6-d83d3452e65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:46.595185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 683/1370] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:46.932904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352f406-b11d-45ea-8900-a00c4a4d12e7 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:46.933292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352f406-b11d-45ea-8900-a00c4a4d12e7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:46.933687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352f406-b11d-45ea-8900-a00c4a4d12e7 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:46.935248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 688/1371] 10.222.0.72 () {54 vars in 964 bytes} [Tue Sep 13 12:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:46.940349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13a73-842c-4bcb-b04c-cc5b92535450 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:46.940638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13a73-842c-4bcb-b04c-cc5b92535450 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:46.940914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13a73-842c-4bcb-b04c-cc5b92535450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:47.022853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-20a13a73-842c-4bcb-b04c-cc5b92535450 None None] MFA Rules not processed for user `6c5b4fdb65514ca6b8033f0f1e4c5eab`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:47.024067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-20a13a73-842c-4bcb-b04c-cc5b92535450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:47.076178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:47.114037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 684/1372] 10.222.0.72 () {58 vars in 1061 bytes} [Tue Sep 13 12:57:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:47.119677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352f406-b11d-45ea-8900-a00c4a4d12e7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:47.121432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c352f406-b11d-45ea-8900-a00c4a4d12e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:47.176210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:47.234933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352f406-b11d-45ea-8900-a00c4a4d12e7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:47.320857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e3404-f28b-498f-914f-7a02a1adcacb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:47.321523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e3404-f28b-498f-914f-7a02a1adcacb service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:47.321823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e3404-f28b-498f-914f-7a02a1adcacb service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:47.322635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e3404-f28b-498f-914f-7a02a1adcacb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:47.324238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ab1e3404-f28b-498f-914f-7a02a1adcacb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:47.429286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:47.442827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e3404-f28b-498f-914f-7a02a1adcacb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:47.444257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e3404-f28b-498f-914f-7a02a1adcacb service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:47.469329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 689/1373] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 12:57:48.805864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5764ee2-afd4-4fb9-9715-23b3fa99c3c2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:48.806461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5764ee2-afd4-4fb9-9715-23b3fa99c3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:48.806461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5764ee2-afd4-4fb9-9715-23b3fa99c3c2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:48.808927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 685/1374] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:48.814204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c999af0-0d39-4173-bd60-41ee1aee0d91 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:48.814934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c999af0-0d39-4173-bd60-41ee1aee0d91 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:48.815209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c999af0-0d39-4173-bd60-41ee1aee0d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:48.881917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4c999af0-0d39-4173-bd60-41ee1aee0d91 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:48.883131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4c999af0-0d39-4173-bd60-41ee1aee0d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:48.984483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 690/1375] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:49.282522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5764ee2-afd4-4fb9-9715-23b3fa99c3c2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:49.284074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b5764ee2-afd4-4fb9-9715-23b3fa99c3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:49.438566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5764ee2-afd4-4fb9-9715-23b3fa99c3c2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:49.492530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:49.558959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87bc46-9630-493a-b813-65c7f1dd4a1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:49.559572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87bc46-9630-493a-b813-65c7f1dd4a1a service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:49.559884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87bc46-9630-493a-b813-65c7f1dd4a1a service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:49.560155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87bc46-9630-493a-b813-65c7f1dd4a1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:49.561882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1b87bc46-9630-493a-b813-65c7f1dd4a1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:49.659457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87bc46-9630-493a-b813-65c7f1dd4a1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:49.661112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87bc46-9630-493a-b813-65c7f1dd4a1a service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:49.679765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 686/1376] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:52.026569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e721-06ad-46c3-ab8b-eb51afddb57c None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:52.027117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e721-06ad-46c3-ab8b-eb51afddb57c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:52.027117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637e721-06ad-46c3-ab8b-eb51afddb57c None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:52.028602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 691/1377] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:52.033688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583532f-cff2-420e-bbf2-56b19f0072ea None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:52.034126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583532f-cff2-420e-bbf2-56b19f0072ea None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:52.034314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583532f-cff2-420e-bbf2-56b19f0072ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:52.101091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7583532f-cff2-420e-bbf2-56b19f0072ea None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:52.102360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7583532f-cff2-420e-bbf2-56b19f0072ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:52.110285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:52.205881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 687/1378] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:52.503000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637e721-06ad-46c3-ab8b-eb51afddb57c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:52.504673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2637e721-06ad-46c3-ab8b-eb51afddb57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:52.579454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:52.681925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637e721-06ad-46c3-ab8b-eb51afddb57c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:52.809875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39544d-f1eb-4062-b926-0b200c283062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:52.810790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39544d-f1eb-4062-b926-0b200c283062 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:52.811220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39544d-f1eb-4062-b926-0b200c283062 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:52.811643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39544d-f1eb-4062-b926-0b200c283062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:52.813422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4b39544d-f1eb-4062-b926-0b200c283062 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:52.838984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:52.918201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39544d-f1eb-4062-b926-0b200c283062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:52.919965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39544d-f1eb-4062-b926-0b200c283062 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:52.940647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 692/1379] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:55.329823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8f80-6612-48ee-b2b3-0b0c230d3efc None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:55.330450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8f80-6612-48ee-b2b3-0b0c230d3efc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:55.330450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8f80-6612-48ee-b2b3-0b0c230d3efc None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:55.332994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 688/1380] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:55.337289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3199b7-32a0-40bb-9240-7fdc27444780 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:55.337840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3199b7-32a0-40bb-9240-7fdc27444780 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:55.338042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3199b7-32a0-40bb-9240-7fdc27444780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:55.408210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-fb3199b7-32a0-40bb-9240-7fdc27444780 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:55.409189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fb3199b7-32a0-40bb-9240-7fdc27444780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:55.518149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:55.523959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 693/1381] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:57:55.814173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8f80-6612-48ee-b2b3-0b0c230d3efc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:55.815716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f3ea8f80-6612-48ee-b2b3-0b0c230d3efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:55.954558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:55.965072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8f80-6612-48ee-b2b3-0b0c230d3efc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:56.087106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48f1bd-82b6-40f9-826d-a52702157875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:56.088239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f1bd-82b6-40f9-826d-a52702157875 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:56.088592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f1bd-82b6-40f9-826d-a52702157875 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:56.088948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f1bd-82b6-40f9-826d-a52702157875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:56.091057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8b48f1bd-82b6-40f9-826d-a52702157875 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:56.186842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48f1bd-82b6-40f9-826d-a52702157875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:56.188441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48f1bd-82b6-40f9-826d-a52702157875 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:56.199316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:56.207486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 689/1382] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:57:58.575406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a9ae4-419a-480f-a5cd-2d9b396e2a6a None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:58.576204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a9ae4-419a-480f-a5cd-2d9b396e2a6a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:58.576204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a9ae4-419a-480f-a5cd-2d9b396e2a6a None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:58.577329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 694/1383] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:57:58.583132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1092945-ffab-4813-a4ee-cae8da1a38d2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:58.583626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1092945-ffab-4813-a4ee-cae8da1a38d2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:58.583787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1092945-ffab-4813-a4ee-cae8da1a38d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:58.652245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b1092945-ffab-4813-a4ee-cae8da1a38d2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:57:58.653468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b1092945-ffab-4813-a4ee-cae8da1a38d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:58.754250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 690/1384] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:57:59.040403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a9ae4-419a-480f-a5cd-2d9b396e2a6a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:57:59.042649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ff7a9ae4-419a-480f-a5cd-2d9b396e2a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:59.193454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a9ae4-419a-480f-a5cd-2d9b396e2a6a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:57:59.262072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:57:59.310645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b455f-a8ec-4a9b-b406-05805dd00085 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:57:59.311181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b455f-a8ec-4a9b-b406-05805dd00085 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:57:59.311463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b455f-a8ec-4a9b-b406-05805dd00085 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:57:59.311740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b455f-a8ec-4a9b-b406-05805dd00085 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:57:59.313343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f45b455f-a8ec-4a9b-b406-05805dd00085 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:57:59.410645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b455f-a8ec-4a9b-b406-05805dd00085 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:57:59.412267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b455f-a8ec-4a9b-b406-05805dd00085 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:57:59.431249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 695/1385] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:01.647523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5008f2-6103-4455-a604-0d702b1ca863 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:01.647995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5008f2-6103-4455-a604-0d702b1ca863 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:01.648093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5008f2-6103-4455-a604-0d702b1ca863 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:01.649408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 691/1386] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:01.655391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9db7cf-bc73-466e-96e7-113c286cfe9b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:01.655823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9db7cf-bc73-466e-96e7-113c286cfe9b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:01.655969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9db7cf-bc73-466e-96e7-113c286cfe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:01.722993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1c9db7cf-bc73-466e-96e7-113c286cfe9b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:01.724202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1c9db7cf-bc73-466e-96e7-113c286cfe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:01.758235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:01.823262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 696/1387] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:02.110519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5008f2-6103-4455-a604-0d702b1ca863 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:02.112045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5d5008f2-6103-4455-a604-0d702b1ca863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:02.188699 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:02.258692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5008f2-6103-4455-a604-0d702b1ca863 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:02.373096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7162fc3-d88a-444e-9db4-cf41d6b5d112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:02.373625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7162fc3-d88a-444e-9db4-cf41d6b5d112 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:02.373929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7162fc3-d88a-444e-9db4-cf41d6b5d112 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:02.374199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7162fc3-d88a-444e-9db4-cf41d6b5d112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:02.375781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c7162fc3-d88a-444e-9db4-cf41d6b5d112 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:02.413869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:02.465794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7162fc3-d88a-444e-9db4-cf41d6b5d112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:02.467371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7162fc3-d88a-444e-9db4-cf41d6b5d112 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:02.487966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 692/1388] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:03.177763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba6ff8-1e49-4eca-8819-bebc52c9546b None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:03.178325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba6ff8-1e49-4eca-8819-bebc52c9546b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:03.178325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba6ff8-1e49-4eca-8819-bebc52c9546b None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:03.179759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 697/1389] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:03.190368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa145ed-ee55-4b6f-a2a8-a3b7642f69d1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:03.190368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa145ed-ee55-4b6f-a2a8-a3b7642f69d1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:03.190368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa145ed-ee55-4b6f-a2a8-a3b7642f69d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:03.255498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-caa145ed-ee55-4b6f-a2a8-a3b7642f69d1 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:03.256704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-caa145ed-ee55-4b6f-a2a8-a3b7642f69d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:03.357342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 693/1390] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:03.662399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d316b-87fd-4d4f-b92e-08da8b5511c2 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:03.662783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d316b-87fd-4d4f-b92e-08da8b5511c2 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:03.663060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d316b-87fd-4d4f-b92e-08da8b5511c2 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:03.664406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 698/1391] 10.222.0.72 () {54 vars in 966 bytes} [Tue Sep 13 12:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:03.670648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a02ba4-0b5f-42e4-be9f-a44c1406c49c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:03.670993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a02ba4-0b5f-42e4-be9f-a44c1406c49c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:03.671273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a02ba4-0b5f-42e4-be9f-a44c1406c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:03.687313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:03.748075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-66a02ba4-0b5f-42e4-be9f-a44c1406c49c None None] MFA Rules not processed for user `04084ffe1b0548fea75ea144977e4f15`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:03.749290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-66a02ba4-0b5f-42e4-be9f-a44c1406c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:03.826466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 694/1392] 10.222.0.72 () {58 vars in 1063 bytes} [Tue Sep 13 12:58:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:03.835234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d316b-87fd-4d4f-b92e-08da8b5511c2 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:03.836736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f62d316b-87fd-4d4f-b92e-08da8b5511c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:03.944115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d316b-87fd-4d4f-b92e-08da8b5511c2 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:04.002563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:04.018171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bace9-7742-4432-9d36-26b104fd7467 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04084ffe1b0548fea75ea144977e4f15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:04.025610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bace9-7742-4432-9d36-26b104fd7467 service glance] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:04.025610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bace9-7742-4432-9d36-26b104fd7467 service glance] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:04.025610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bace9-7742-4432-9d36-26b104fd7467 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:04.025610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cd9bace9-7742-4432-9d36-26b104fd7467 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:04.129220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bace9-7742-4432-9d36-26b104fd7467 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:04.130734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bace9-7742-4432-9d36-26b104fd7467 service glance] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:04.149423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 699/1393] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:04.891775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6c251-84ad-4559-996c-f8642676c665 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:04.892253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6c251-84ad-4559-996c-f8642676c665 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:04.892368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6c251-84ad-4559-996c-f8642676c665 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:04.893837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 695/1394] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:04.899094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aedf64-aaaa-49d5-ad6b-e67509d90f64 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:04.899490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aedf64-aaaa-49d5-ad6b-e67509d90f64 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:04.900163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aedf64-aaaa-49d5-ad6b-e67509d90f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:04.965759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-04aedf64-aaaa-49d5-ad6b-e67509d90f64 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:04.967036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-04aedf64-aaaa-49d5-ad6b-e67509d90f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:05.016391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:05.022626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fbd7-46da-4c68-ac58-8179d770d00a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:05.022626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fbd7-46da-4c68-ac58-8179d770d00a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:05.022949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8fbd7-46da-4c68-ac58-8179d770d00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:05.080835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 700/1395] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:05.101146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0fd8fbd7-46da-4c68-ac58-8179d770d00a None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:05.102544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0fd8fbd7-46da-4c68-ac58-8179d770d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:05.160181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:05.193086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 696/1396] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 12:58:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:05.237016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b48012-0371-4dbb-ac2d-d37192ab70ab None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:05.237598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b48012-0371-4dbb-ac2d-d37192ab70ab None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:05.237914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b48012-0371-4dbb-ac2d-d37192ab70ab None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:05.239279 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 701/1397] 10.222.0.72 () {54 vars in 943 bytes} [Tue Sep 13 12:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:05.246143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5d9f-1768-4002-a664-9be1cdb2c9c2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:05.249552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5d9f-1768-4002-a664-9be1cdb2c9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:05.249552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5d9f-1768-4002-a664-9be1cdb2c9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:05.327248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-995f5d9f-1768-4002-a664-9be1cdb2c9c2 None None] MFA Rules not processed for user `a08e9745841d4716a64700da59eebf04`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:05.328795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-995f5d9f-1768-4002-a664-9be1cdb2c9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:05.379826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b48012-0371-4dbb-ac2d-d37192ab70ab None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:05.381372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-89b48012-0371-4dbb-ac2d-d37192ab70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:05.439866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 697/1398] 10.222.0.72 () {58 vars in 1040 bytes} [Tue Sep 13 12:58:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:58:05.450174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5d9f-1768-4002-a664-9be1cdb2c9c2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:05.459561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5d9f-1768-4002-a664-9be1cdb2c9c2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:05.487178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:05.535706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b48012-0371-4dbb-ac2d-d37192ab70ab None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:05.562902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:05.568505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2190b1-3da6-4f35-b0d9-2dc62f83f191 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:05.569112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2190b1-3da6-4f35-b0d9-2dc62f83f191 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:05.569385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2190b1-3da6-4f35-b0d9-2dc62f83f191 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:05.569654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2190b1-3da6-4f35-b0d9-2dc62f83f191 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:05.571378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ee2190b1-3da6-4f35-b0d9-2dc62f83f191 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:05.656946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a081a14-3e69-4363-a837-72c9beb27941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:05.657772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a081a14-3e69-4363-a837-72c9beb27941 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:05.658052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a081a14-3e69-4363-a837-72c9beb27941 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:05.658326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a081a14-3e69-4363-a837-72c9beb27941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:05.660441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9a081a14-3e69-4363-a837-72c9beb27941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:05.660756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2190b1-3da6-4f35-b0d9-2dc62f83f191 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:05.662393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2190b1-3da6-4f35-b0d9-2dc62f83f191 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:05.670707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 698/1399] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 12:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 12:58:05.750860 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a081a14-3e69-4363-a837-72c9beb27941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:05.752452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a081a14-3e69-4363-a837-72c9beb27941 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:05.770339 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 702/1400] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 12:58:08.186152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cefa18-45be-4661-8a69-00fa7290f1ca None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:08.186728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cefa18-45be-4661-8a69-00fa7290f1ca None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:08.186728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cefa18-45be-4661-8a69-00fa7290f1ca None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:08.188325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 699/1401] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:08.193823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9643-3fe8-46db-aab1-d74bf1b45ab5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:08.194321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9643-3fe8-46db-aab1-d74bf1b45ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:08.194412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9643-3fe8-46db-aab1-d74bf1b45ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:08.234549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:08.271856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-637b9643-3fe8-46db-aab1-d74bf1b45ab5 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:08.273173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-637b9643-3fe8-46db-aab1-d74bf1b45ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:08.385434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 703/1402] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:08.675638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cefa18-45be-4661-8a69-00fa7290f1ca None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:08.677256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-98cefa18-45be-4661-8a69-00fa7290f1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:08.748458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:08.844001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cefa18-45be-4661-8a69-00fa7290f1ca None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:08.973241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63d83a-baef-4500-94f6-427b8347eed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:08.973893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d83a-baef-4500-94f6-427b8347eed5 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:08.974170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d83a-baef-4500-94f6-427b8347eed5 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:08.975238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d83a-baef-4500-94f6-427b8347eed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:08.976866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8a63d83a-baef-4500-94f6-427b8347eed5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:09.002161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:09.082465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63d83a-baef-4500-94f6-427b8347eed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:09.084207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63d83a-baef-4500-94f6-427b8347eed5 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:09.103870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 700/1403] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:11.521667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541ad53-1268-4b6c-9884-5461e738fcf0 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:11.522258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541ad53-1268-4b6c-9884-5461e738fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:11.522356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541ad53-1268-4b6c-9884-5461e738fcf0 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:11.523670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 704/1404] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:11.529257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90160cd-c2bb-4934-86bf-80a0154ef9b3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:11.529691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90160cd-c2bb-4934-86bf-80a0154ef9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:11.529926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90160cd-c2bb-4934-86bf-80a0154ef9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:11.606594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a90160cd-c2bb-4934-86bf-80a0154ef9b3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:11.611054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a90160cd-c2bb-4934-86bf-80a0154ef9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:11.720116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:11.733887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 701/1405] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:12.023726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541ad53-1268-4b6c-9884-5461e738fcf0 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:12.025804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5541ad53-1268-4b6c-9884-5461e738fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:12.147558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:12.191266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541ad53-1268-4b6c-9884-5461e738fcf0 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:12.322448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb41f5-4783-4bdf-bbd7-b0b5686137e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:12.323326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb41f5-4783-4bdf-bbd7-b0b5686137e6 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:12.323606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb41f5-4783-4bdf-bbd7-b0b5686137e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:12.323986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb41f5-4783-4bdf-bbd7-b0b5686137e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:12.325461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-21fb41f5-4783-4bdf-bbd7-b0b5686137e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:12.407291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:12.433972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb41f5-4783-4bdf-bbd7-b0b5686137e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:12.435853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb41f5-4783-4bdf-bbd7-b0b5686137e6 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:12.466062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 705/1406] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:14.877264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41564d7-2464-44bd-bddc-68378b7ceecc None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:14.877822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41564d7-2464-44bd-bddc-68378b7ceecc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:14.877822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41564d7-2464-44bd-bddc-68378b7ceecc None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:14.879078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 702/1407] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:14.884897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf175f-d9a9-4479-8d4b-2f57de8c95be None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:14.885341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf175f-d9a9-4479-8d4b-2f57de8c95be None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:14.885458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf175f-d9a9-4479-8d4b-2f57de8c95be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:14.964086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5baf175f-d9a9-4479-8d4b-2f57de8c95be None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:14.965623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5baf175f-d9a9-4479-8d4b-2f57de8c95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:15.084358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 706/1408] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:15.374947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41564d7-2464-44bd-bddc-68378b7ceecc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:15.376594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e41564d7-2464-44bd-bddc-68378b7ceecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:15.538620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41564d7-2464-44bd-bddc-68378b7ceecc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:15.620771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:15.678878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6103-0cdc-41ab-818b-da17831ff3f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:15.679504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6103-0cdc-41ab-818b-da17831ff3f1 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:15.679779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6103-0cdc-41ab-818b-da17831ff3f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:15.680053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6103-0cdc-41ab-818b-da17831ff3f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:15.682590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c87b6103-0cdc-41ab-818b-da17831ff3f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:15.787613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6103-0cdc-41ab-818b-da17831ff3f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:15.789327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6103-0cdc-41ab-818b-da17831ff3f1 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:15.810189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 703/1409] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:25.082833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=79 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:25.807058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=64 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:29.838595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8ac01-3e5b-4dff-918d-87471d509cc0 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:29.839164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8ac01-3e5b-4dff-918d-87471d509cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:29.839164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8ac01-3e5b-4dff-918d-87471d509cc0 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:29.840639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 707/1410] 10.222.0.72 () {54 vars in 918 bytes} [Tue Sep 13 12:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:29.845776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7b93-ad83-4d75-9f6f-a8c01c865e1e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:29.846201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7b93-ad83-4d75-9f6f-a8c01c865e1e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:29.846358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7b93-ad83-4d75-9f6f-a8c01c865e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:29.923416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-24cb7b93-ad83-4d75-9f6f-a8c01c865e1e None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:29.926449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-24cb7b93-ad83-4d75-9f6f-a8c01c865e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:30.030125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 704/1411] 10.222.0.72 () {58 vars in 1015 bytes} [Tue Sep 13 12:58:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:30.045086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a476-f42b-4410-a240-9233f0df7fea None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:30.045368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a476-f42b-4410-a240-9233f0df7fea None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:30.045684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a476-f42b-4410-a240-9233f0df7fea None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:30.047009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 708/1412] 10.222.0.72 () {54 vars in 918 bytes} [Tue Sep 13 12:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:30.051593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696e5c6-b265-4893-a0c5-bf5feaa42788 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:30.051880 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696e5c6-b265-4893-a0c5-bf5feaa42788 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:30.052153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696e5c6-b265-4893-a0c5-bf5feaa42788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:30.125935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:30.128477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c696e5c6-b265-4893-a0c5-bf5feaa42788 None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:30.129686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c696e5c6-b265-4893-a0c5-bf5feaa42788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:30.245084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 705/1413] 10.222.0.72 () {58 vars in 1015 bytes} [Tue Sep 13 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:32.831134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cf53e-bca5-4f22-8362-eb070870343e None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:32.831665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cf53e-bca5-4f22-8362-eb070870343e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:32.831665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cf53e-bca5-4f22-8362-eb070870343e None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:32.832998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:32.838372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c62d7-24ff-4da4-887b-1d17bb1ed5c4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:32.838802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c62d7-24ff-4da4-887b-1d17bb1ed5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:32.838953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c62d7-24ff-4da4-887b-1d17bb1ed5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:32.904153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-528c62d7-24ff-4da4-887b-1d17bb1ed5c4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:32.905275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-528c62d7-24ff-4da4-887b-1d17bb1ed5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:32.959883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:33.001960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:33.298995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cf53e-bca5-4f22-8362-eb070870343e None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:33.300523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6a0cf53e-bca5-4f22-8362-eb070870343e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:33.449479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cf53e-bca5-4f22-8362-eb070870343e None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:33.520940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:33.562928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8028a-cae4-40c1-8ea9-95fd4db1fb5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:33.563535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8028a-cae4-40c1-8ea9-95fd4db1fb5e service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:33.563814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8028a-cae4-40c1-8ea9-95fd4db1fb5e service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:33.564088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8028a-cae4-40c1-8ea9-95fd4db1fb5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:33.565682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-05f8028a-cae4-40c1-8ea9-95fd4db1fb5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:33.654870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8028a-cae4-40c1-8ea9-95fd4db1fb5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:33.656473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8028a-cae4-40c1-8ea9-95fd4db1fb5e service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:33.674461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 710/1416] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:34.812140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92421-be93-4c24-89a6-e07b005382f8 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:34.812600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92421-be93-4c24-89a6-e07b005382f8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:34.812708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d92421-be93-4c24-89a6-e07b005382f8 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:34.814054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:34.820256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e7e64-8ed8-408b-a44f-1ed2d908c589 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:34.820675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e7e64-8ed8-408b-a44f-1ed2d908c589 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:34.820800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e7e64-8ed8-408b-a44f-1ed2d908c589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:34.895325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-985e7e64-8ed8-408b-a44f-1ed2d908c589 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:34.896455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-985e7e64-8ed8-408b-a44f-1ed2d908c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:34.923799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:34.992204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:35.141461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92421-be93-4c24-89a6-e07b005382f8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:35.143023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-66d92421-be93-4c24-89a6-e07b005382f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:35.286130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d92421-be93-4c24-89a6-e07b005382f8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:35.327359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:35.401689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02a6c9-95f6-455a-a3a1-f369b273da99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:35.402321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02a6c9-95f6-455a-a3a1-f369b273da99 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:35.402602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02a6c9-95f6-455a-a3a1-f369b273da99 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:35.403462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02a6c9-95f6-455a-a3a1-f369b273da99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:35.405090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5d02a6c9-95f6-455a-a3a1-f369b273da99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:35.495066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02a6c9-95f6-455a-a3a1-f369b273da99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:35.496662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02a6c9-95f6-455a-a3a1-f369b273da99 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:35.514405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 708/1419] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:36.642058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8262be2-c8ca-4892-a574-883c973cb5c8 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:36.642528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8262be2-c8ca-4892-a574-883c973cb5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:36.642625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8262be2-c8ca-4892-a574-883c973cb5c8 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:36.643988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:36.651336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e52a0-d98a-467f-a6fc-ab554dc51474 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:36.651769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e52a0-d98a-467f-a6fc-ab554dc51474 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:36.651888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e52a0-d98a-467f-a6fc-ab554dc51474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:36.706378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:36.727458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ca1e52a0-d98a-467f-a6fc-ab554dc51474 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:36.728560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ca1e52a0-d98a-467f-a6fc-ab554dc51474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:36.832095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:36.982224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8262be2-c8ca-4892-a574-883c973cb5c8 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:36.983766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a8262be2-c8ca-4892-a574-883c973cb5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:37.129434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8262be2-c8ca-4892-a574-883c973cb5c8 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:37.141282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:37.243987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7177b7-65c6-45d4-8bf8-b6495c63214c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:37.244534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7177b7-65c6-45d4-8bf8-b6495c63214c service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:37.244813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7177b7-65c6-45d4-8bf8-b6495c63214c service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:37.245086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7177b7-65c6-45d4-8bf8-b6495c63214c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:37.246703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bb7177b7-65c6-45d4-8bf8-b6495c63214c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:37.335883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7177b7-65c6-45d4-8bf8-b6495c63214c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:37.338634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7177b7-65c6-45d4-8bf8-b6495c63214c service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:37.357299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 713/1422] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:38.470669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a34de-8932-43b0-aef8-658e7d5393fe None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:38.471221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a34de-8932-43b0-aef8-658e7d5393fe None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:38.471221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a34de-8932-43b0-aef8-658e7d5393fe None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:38.472602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:38.479581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef81500-fb96-43b5-b8ed-3347509ea236 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:38.480029 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef81500-fb96-43b5-b8ed-3347509ea236 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:38.480132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef81500-fb96-43b5-b8ed-3347509ea236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:38.499990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:38.556626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6ef81500-fb96-43b5-b8ed-3347509ea236 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:38.557868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6ef81500-fb96-43b5-b8ed-3347509ea236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:38.654744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:58:38.804831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a34de-8932-43b0-aef8-658e7d5393fe None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:38.806799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f86a34de-8932-43b0-aef8-658e7d5393fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:38.916548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:38.952187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a34de-8932-43b0-aef8-658e7d5393fe None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:39.064944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40510b5f-f35f-4179-8cd3-b392373b97f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:39.065464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510b5f-f35f-4179-8cd3-b392373b97f2 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:39.065768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510b5f-f35f-4179-8cd3-b392373b97f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:39.066047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510b5f-f35f-4179-8cd3-b392373b97f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:39.067653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-40510b5f-f35f-4179-8cd3-b392373b97f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:39.144432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:39.158003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40510b5f-f35f-4179-8cd3-b392373b97f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:39.159603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40510b5f-f35f-4179-8cd3-b392373b97f2 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:39.177275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 711/1425] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:40.297404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f3d5-c78d-4a71-88b1-003d00ed881a None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:40.298068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f3d5-c78d-4a71-88b1-003d00ed881a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:40.298068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5f3d5-c78d-4a71-88b1-003d00ed881a None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:40.299367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:40.306163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2826f18-512b-4643-ae9d-a81a6022fc39 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:40.306630 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2826f18-512b-4643-ae9d-a81a6022fc39 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:40.306745 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2826f18-512b-4643-ae9d-a81a6022fc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:40.384274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b2826f18-512b-4643-ae9d-a81a6022fc39 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:40.385862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b2826f18-512b-4643-ae9d-a81a6022fc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:40.481872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:40.630936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5f3d5-c78d-4a71-88b1-003d00ed881a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:40.632425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9ce5f3d5-c78d-4a71-88b1-003d00ed881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:40.716530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:40.778857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5f3d5-c78d-4a71-88b1-003d00ed881a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:40.891758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad99dbd-defc-41cd-a156-c6c00eedef95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:40.892277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad99dbd-defc-41cd-a156-c6c00eedef95 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:40.892551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad99dbd-defc-41cd-a156-c6c00eedef95 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:40.892822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad99dbd-defc-41cd-a156-c6c00eedef95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:40.894445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0ad99dbd-defc-41cd-a156-c6c00eedef95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:40.941861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:40.982430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad99dbd-defc-41cd-a156-c6c00eedef95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:40.984009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad99dbd-defc-41cd-a156-c6c00eedef95 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:41.002141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 716/1428] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:42.122383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebb94-5f9c-4528-82f0-c547165d2ead None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:42.122945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebb94-5f9c-4528-82f0-c547165d2ead None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:42.122945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ebb94-5f9c-4528-82f0-c547165d2ead None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:42.124266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:42.130644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c887-4122-4b9e-b4a9-e7e20f79710c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:42.131058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c887-4122-4b9e-b4a9-e7e20f79710c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:42.131228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c887-4122-4b9e-b4a9-e7e20f79710c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:42.207014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f243c887-4122-4b9e-b4a9-e7e20f79710c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:42.208256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f243c887-4122-4b9e-b4a9-e7e20f79710c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:42.304228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:42.456251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebb94-5f9c-4528-82f0-c547165d2ead None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:42.457889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4a0ebb94-5f9c-4528-82f0-c547165d2ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:42.505702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:42.599684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ebb94-5f9c-4528-82f0-c547165d2ead None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:42.711812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354d869-3834-4bbc-9c6e-e96c8ddc5b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:42.712344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354d869-3834-4bbc-9c6e-e96c8ddc5b06 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:42.712625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354d869-3834-4bbc-9c6e-e96c8ddc5b06 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:42.712900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354d869-3834-4bbc-9c6e-e96c8ddc5b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:42.714549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8354d869-3834-4bbc-9c6e-e96c8ddc5b06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:42.721610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:42.804011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354d869-3834-4bbc-9c6e-e96c8ddc5b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:42.807291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354d869-3834-4bbc-9c6e-e96c8ddc5b06 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:42.825129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 714/1431] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 12:58:43.955146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c9db-02d5-456f-8465-46826551d19f None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:43.955623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c9db-02d5-456f-8465-46826551d19f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:43.955719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c9db-02d5-456f-8465-46826551d19f None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:43.957009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:43.963982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edae34-6051-4305-aaba-a67a21f90058 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:43.964416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edae34-6051-4305-aaba-a67a21f90058 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:43.964528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edae34-6051-4305-aaba-a67a21f90058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:44.042209 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-84edae34-6051-4305-aaba-a67a21f90058 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:44.043445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-84edae34-6051-4305-aaba-a67a21f90058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:44.111656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:44.141214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:44.293067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658c9db-02d5-456f-8465-46826551d19f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:44.294623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e658c9db-02d5-456f-8465-46826551d19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:44.312291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:44.439994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658c9db-02d5-456f-8465-46826551d19f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:44.526867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:44.551580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945d141-5230-4503-a602-1ae280a850a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:44.552182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945d141-5230-4503-a602-1ae280a850a9 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:44.552455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945d141-5230-4503-a602-1ae280a850a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:44.552724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945d141-5230-4503-a602-1ae280a850a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:44.554350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6945d141-5230-4503-a602-1ae280a850a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:44.641558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945d141-5230-4503-a602-1ae280a850a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:44.643196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945d141-5230-4503-a602-1ae280a850a9 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:44.661246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 719/1434] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 12:58:45.829537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3ea41-6092-462c-ba31-1c64e334ef99 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:45.830133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3ea41-6092-462c-ba31-1c64e334ef99 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:45.830133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3ea41-6092-462c-ba31-1c64e334ef99 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:45.831487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:45.838363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431e08-d9fe-4739-a242-a43b1051a58e None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:45.838789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431e08-d9fe-4739-a242-a43b1051a58e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:45.838917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f431e08-d9fe-4739-a242-a43b1051a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:45.917759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-3f431e08-d9fe-4739-a242-a43b1051a58e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:45.918974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3f431e08-d9fe-4739-a242-a43b1051a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:45.959966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:46.013041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:46.163319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3ea41-6092-462c-ba31-1c64e334ef99 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:46.165505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d0f3ea41-6092-462c-ba31-1c64e334ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:46.309981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3ea41-6092-462c-ba31-1c64e334ef99 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:46.366540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:46.428176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427c9c0-e788-4363-a29d-92e6b0bd429f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:46.428698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427c9c0-e788-4363-a29d-92e6b0bd429f service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:46.428976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427c9c0-e788-4363-a29d-92e6b0bd429f service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:46.429247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427c9c0-e788-4363-a29d-92e6b0bd429f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:46.431204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f427c9c0-e788-4363-a29d-92e6b0bd429f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:46.637932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427c9c0-e788-4363-a29d-92e6b0bd429f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:46.639549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427c9c0-e788-4363-a29d-92e6b0bd429f service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:46.658290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 717/1437] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:47.774463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875306e-d221-4aa7-af9f-557fdfca9c10 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:47.775183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875306e-d221-4aa7-af9f-557fdfca9c10 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:47.775183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875306e-d221-4aa7-af9f-557fdfca9c10 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:47.776364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 12:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:47.782870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19448e30-ffec-4676-bd7c-2ae5d3e15bde None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:47.783324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19448e30-ffec-4676-bd7c-2ae5d3e15bde None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:47.783435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19448e30-ffec-4676-bd7c-2ae5d3e15bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:47.859156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-19448e30-ffec-4676-bd7c-2ae5d3e15bde None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:47.860369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-19448e30-ffec-4676-bd7c-2ae5d3e15bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:47.862812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:47.955447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 12:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:48.106518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875306e-d221-4aa7-af9f-557fdfca9c10 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:48.108059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1875306e-d221-4aa7-af9f-557fdfca9c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:48.250569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875306e-d221-4aa7-af9f-557fdfca9c10 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:48.280214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:48.369141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032368f-b651-4c19-b42a-0e4ea7cd64e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:48.369783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032368f-b651-4c19-b42a-0e4ea7cd64e7 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:48.370089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032368f-b651-4c19-b42a-0e4ea7cd64e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:48.370365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032368f-b651-4c19-b42a-0e4ea7cd64e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:48.371980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5032368f-b651-4c19-b42a-0e4ea7cd64e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:48.463767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032368f-b651-4c19-b42a-0e4ea7cd64e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:48.465375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032368f-b651-4c19-b42a-0e4ea7cd64e7 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:48.483754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 722/1440] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:50.562869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e90e-43cb-4208-852f-8ffc097a2459 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:50.563436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e90e-43cb-4208-852f-8ffc097a2459 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:50.563436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22e90e-43cb-4208-852f-8ffc097a2459 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:50.564809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:50.570161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70820f5a-07ce-4507-a527-db8609d41425 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:50.570568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70820f5a-07ce-4507-a527-db8609d41425 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:50.570732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70820f5a-07ce-4507-a527-db8609d41425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:50.610895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:50.636396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-70820f5a-07ce-4507-a527-db8609d41425 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:50.637622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-70820f5a-07ce-4507-a527-db8609d41425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:50.734323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:58:51.020015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22e90e-43cb-4208-852f-8ffc097a2459 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:51.021574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1d22e90e-43cb-4208-852f-8ffc097a2459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:51.123195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22e90e-43cb-4208-852f-8ffc097a2459 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:51.154655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:51.198116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0b4f-42f2-41ee-bf92-aeffc08086c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:51.198722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0b4f-42f2-41ee-bf92-aeffc08086c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:51.199004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0b4f-42f2-41ee-bf92-aeffc08086c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:51.199278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0b4f-42f2-41ee-bf92-aeffc08086c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:51.200872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-145b0b4f-42f2-41ee-bf92-aeffc08086c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:51.311852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b0b4f-42f2-41ee-bf92-aeffc08086c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:51.313204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b0b4f-42f2-41ee-bf92-aeffc08086c9 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:51.331547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 720/1443] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:53.691109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545f939-dbbf-4a12-8c3d-2a9b6c754765 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:53.691672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545f939-dbbf-4a12-8c3d-2a9b6c754765 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:53.691672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545f939-dbbf-4a12-8c3d-2a9b6c754765 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:53.693075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:53.699123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f87e4-2334-4d2c-800e-f7e2fc5a2ce0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:53.699813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f87e4-2334-4d2c-800e-f7e2fc5a2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:53.700025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f87e4-2334-4d2c-800e-f7e2fc5a2ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:53.765953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-240f87e4-2334-4d2c-800e-f7e2fc5a2ce0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:53.767152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-240f87e4-2334-4d2c-800e-f7e2fc5a2ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:53.779939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:53.861661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:58:54.150592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545f939-dbbf-4a12-8c3d-2a9b6c754765 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:54.152193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4545f939-dbbf-4a12-8c3d-2a9b6c754765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:54.256364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545f939-dbbf-4a12-8c3d-2a9b6c754765 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:54.268963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:54.331287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e56883-7b34-4f74-8b76-4712e155dc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:54.331956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e56883-7b34-4f74-8b76-4712e155dc2a service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:54.332239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e56883-7b34-4f74-8b76-4712e155dc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:54.332515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e56883-7b34-4f74-8b76-4712e155dc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:54.334189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d4e56883-7b34-4f74-8b76-4712e155dc2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:54.447648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e56883-7b34-4f74-8b76-4712e155dc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:54.449092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e56883-7b34-4f74-8b76-4712e155dc2a service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:54.467299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 725/1446] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:58:56.993311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515ca7e-4831-4dda-a42c-11495f8fc092 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:56.993884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515ca7e-4831-4dda-a42c-11495f8fc092 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:56.993884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515ca7e-4831-4dda-a42c-11495f8fc092 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:56.995210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:58:57.000416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8e536-1715-46d8-bb7b-097b3dfa6896 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:57.000835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8e536-1715-46d8-bb7b-097b3dfa6896 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:57.000970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8e536-1715-46d8-bb7b-097b3dfa6896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:57.048787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:57.066548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-68a8e536-1715-46d8-bb7b-097b3dfa6896 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:58:57.067767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-68a8e536-1715-46d8-bb7b-097b3dfa6896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:57.163199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:58:57.448194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515ca7e-4831-4dda-a42c-11495f8fc092 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:58:57.449721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4515ca7e-4831-4dda-a42c-11495f8fc092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:57.551137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515ca7e-4831-4dda-a42c-11495f8fc092 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:58:57.591075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:58:57.623778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03eeccd-4c8e-47ce-9287-47aa9c4421a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:58:57.624387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eeccd-4c8e-47ce-9287-47aa9c4421a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:58:57.624663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eeccd-4c8e-47ce-9287-47aa9c4421a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:58:57.624931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03eeccd-4c8e-47ce-9287-47aa9c4421a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:58:57.626543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d03eeccd-4c8e-47ce-9287-47aa9c4421a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:58:57.735900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03eeccd-4c8e-47ce-9287-47aa9c4421a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:58:57.737318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03eeccd-4c8e-47ce-9287-47aa9c4421a3 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:58:57.755324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 723/1449] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:00.072527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335128e5-8a69-4ae3-9ddb-c2ddbe5f5056 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:00.073089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335128e5-8a69-4ae3-9ddb-c2ddbe5f5056 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:00.073188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335128e5-8a69-4ae3-9ddb-c2ddbe5f5056 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:00.075262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:59:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:59:00.081299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190692e1-cc2d-4d69-849a-a272904f6e70 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:00.081810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190692e1-cc2d-4d69-849a-a272904f6e70 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:00.082256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190692e1-cc2d-4d69-849a-a272904f6e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:00.149988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-190692e1-cc2d-4d69-849a-a272904f6e70 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:00.151185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-190692e1-cc2d-4d69-849a-a272904f6e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:00.171481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:00.245095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:00.527644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335128e5-8a69-4ae3-9ddb-c2ddbe5f5056 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:00.529209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-335128e5-8a69-4ae3-9ddb-c2ddbe5f5056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:00.638434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335128e5-8a69-4ae3-9ddb-c2ddbe5f5056 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:00.658734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:00.712382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e3bc5-2df5-48cc-a16b-a02da45a4985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:00.712984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e3bc5-2df5-48cc-a16b-a02da45a4985 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:00.713263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e3bc5-2df5-48cc-a16b-a02da45a4985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:00.713537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e3bc5-2df5-48cc-a16b-a02da45a4985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:00.715154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-330e3bc5-2df5-48cc-a16b-a02da45a4985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:00.824955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e3bc5-2df5-48cc-a16b-a02da45a4985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:00.826376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e3bc5-2df5-48cc-a16b-a02da45a4985 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:00.843986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 728/1452] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:03.046763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65223-f1fa-4000-ae41-00c5f44422ac None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:03.047329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65223-f1fa-4000-ae41-00c5f44422ac None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:03.047329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65223-f1fa-4000-ae41-00c5f44422ac None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:03.048675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:03.054062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0b6df-a426-4884-985c-0f9cf2344502 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:03.054478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0b6df-a426-4884-985c-0f9cf2344502 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:03.054612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0b6df-a426-4884-985c-0f9cf2344502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:03.117432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:03.120728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0fa0b6df-a426-4884-985c-0f9cf2344502 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:03.121956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0fa0b6df-a426-4884-985c-0f9cf2344502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:03.220094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:03.507356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf65223-f1fa-4000-ae41-00c5f44422ac None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:03.508959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ecf65223-f1fa-4000-ae41-00c5f44422ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:03.612540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf65223-f1fa-4000-ae41-00c5f44422ac None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:03.663011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:03.688349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec95cb-0008-4911-a5a0-96b0e1aad221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:03.688957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec95cb-0008-4911-a5a0-96b0e1aad221 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:03.689243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec95cb-0008-4911-a5a0-96b0e1aad221 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:03.689514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec95cb-0008-4911-a5a0-96b0e1aad221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:03.691146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6cec95cb-0008-4911-a5a0-96b0e1aad221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:03.802358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec95cb-0008-4911-a5a0-96b0e1aad221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:03.803730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec95cb-0008-4911-a5a0-96b0e1aad221 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:03.822288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 726/1455] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:06.050502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079aa6c-4eb9-4b6a-adea-90d7bf2fa7fa None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:06.051054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079aa6c-4eb9-4b6a-adea-90d7bf2fa7fa None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:06.051054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079aa6c-4eb9-4b6a-adea-90d7bf2fa7fa None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:06.052397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:06.058233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2db9e-507e-4f97-8490-4775f31ea70f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:06.058641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2db9e-507e-4f97-8490-4775f31ea70f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:06.058820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2db9e-507e-4f97-8490-4775f31ea70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:06.123060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6bf2db9e-507e-4f97-8490-4775f31ea70f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:06.124257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6bf2db9e-507e-4f97-8490-4775f31ea70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:06.152995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:06.217648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:06.503143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079aa6c-4eb9-4b6a-adea-90d7bf2fa7fa None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:06.504747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d079aa6c-4eb9-4b6a-adea-90d7bf2fa7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:06.609525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079aa6c-4eb9-4b6a-adea-90d7bf2fa7fa None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:06.640760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:06.686211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7200dad-57f9-406a-a8e5-a3a1f6e08132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:06.686811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7200dad-57f9-406a-a8e5-a3a1f6e08132 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:06.687084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7200dad-57f9-406a-a8e5-a3a1f6e08132 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:06.687348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7200dad-57f9-406a-a8e5-a3a1f6e08132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:06.688927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e7200dad-57f9-406a-a8e5-a3a1f6e08132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:06.800626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7200dad-57f9-406a-a8e5-a3a1f6e08132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:06.802088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7200dad-57f9-406a-a8e5-a3a1f6e08132 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:06.820784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 731/1458] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:09.024818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087f1ab-8d9e-4918-9797-c3b4b2fa428c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:09.025386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087f1ab-8d9e-4918-9797-c3b4b2fa428c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:09.025386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087f1ab-8d9e-4918-9797-c3b4b2fa428c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:09.026768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:09.031984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a40e35-87b5-4591-8779-43915d325e42 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:09.032405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a40e35-87b5-4591-8779-43915d325e42 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:09.032530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a40e35-87b5-4591-8779-43915d325e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:09.100709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-64a40e35-87b5-4591-8779-43915d325e42 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:09.101975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-64a40e35-87b5-4591-8779-43915d325e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:09.111244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:09.198950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:09.484857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087f1ab-8d9e-4918-9797-c3b4b2fa428c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:09.486444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a087f1ab-8d9e-4918-9797-c3b4b2fa428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:09.588588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087f1ab-8d9e-4918-9797-c3b4b2fa428c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:09.643638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:09.661590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e772a-a426-433f-9b3f-291cee37b3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:09.662223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e772a-a426-433f-9b3f-291cee37b3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:09.666738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e772a-a426-433f-9b3f-291cee37b3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:09.666738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e772a-a426-433f-9b3f-291cee37b3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:09.666738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-694e772a-a426-433f-9b3f-291cee37b3b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:09.773414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e772a-a426-433f-9b3f-291cee37b3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:09.775472 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e772a-a426-433f-9b3f-291cee37b3b3 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:09.793488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 729/1461] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:11.985457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4e776-d046-4591-9c22-c1e2f368634c None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:11.986050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4e776-d046-4591-9c22-c1e2f368634c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:11.986156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4e776-d046-4591-9c22-c1e2f368634c None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:11.987480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:11.993375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7ae6-b632-47ef-ba0b-fbf51efc4525 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:11.993845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7ae6-b632-47ef-ba0b-fbf51efc4525 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:11.994137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7ae6-b632-47ef-ba0b-fbf51efc4525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:12.059920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8f2e7ae6-b632-47ef-ba0b-fbf51efc4525 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:12.061125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8f2e7ae6-b632-47ef-ba0b-fbf51efc4525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:12.099027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:12.156462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:12.441363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4e776-d046-4591-9c22-c1e2f368634c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:12.442945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3ea4e776-d046-4591-9c22-c1e2f368634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:12.546949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4e776-d046-4591-9c22-c1e2f368634c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:12.588378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:12.623023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2736194-9f99-4635-9486-973b9049fd62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:12.623631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2736194-9f99-4635-9486-973b9049fd62 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:12.624108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2736194-9f99-4635-9486-973b9049fd62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:12.624530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2736194-9f99-4635-9486-973b9049fd62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:12.626454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e2736194-9f99-4635-9486-973b9049fd62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:12.735501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2736194-9f99-4635-9486-973b9049fd62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:12.736986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2736194-9f99-4635-9486-973b9049fd62 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:12.756744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 734/1464] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:14.924292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451393c-24c8-44c4-b4be-80e5d8216e3e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:14.924852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451393c-24c8-44c4-b4be-80e5d8216e3e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:14.924852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451393c-24c8-44c4-b4be-80e5d8216e3e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:14.926134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:14.931941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7eb57-d976-4d3c-85d5-920844baff96 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:14.932373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7eb57-d976-4d3c-85d5-920844baff96 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:14.932493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7eb57-d976-4d3c-85d5-920844baff96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:14.997515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-88b7eb57-d976-4d3c-85d5-920844baff96 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:14.998757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-88b7eb57-d976-4d3c-85d5-920844baff96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:15.017056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:15.096031 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:15.378912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451393c-24c8-44c4-b4be-80e5d8216e3e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:15.380441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8451393c-24c8-44c4-b4be-80e5d8216e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:15.487506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451393c-24c8-44c4-b4be-80e5d8216e3e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:15.552816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:15.559542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd23fc7-a148-48e7-aff1-5b54d75a4f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:15.566154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd23fc7-a148-48e7-aff1-5b54d75a4f07 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:15.566154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd23fc7-a148-48e7-aff1-5b54d75a4f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:15.566154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd23fc7-a148-48e7-aff1-5b54d75a4f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:15.566154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dfd23fc7-a148-48e7-aff1-5b54d75a4f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:15.672375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd23fc7-a148-48e7-aff1-5b54d75a4f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:15.673838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd23fc7-a148-48e7-aff1-5b54d75a4f07 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:15.692635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 732/1467] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:17.861115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c2af-377f-4bd6-989d-976b4c56beb6 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:17.861651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c2af-377f-4bd6-989d-976b4c56beb6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:17.861651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c2af-377f-4bd6-989d-976b4c56beb6 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:17.863059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 12:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:17.868620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75750b-9237-432f-a606-3569c58b659e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:17.869067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75750b-9237-432f-a606-3569c58b659e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:17.869170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75750b-9237-432f-a606-3569c58b659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:17.936620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fc75750b-9237-432f-a606-3569c58b659e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:17.937924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc75750b-9237-432f-a606-3569c58b659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:17.987841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:18.038508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:18.330058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c2af-377f-4bd6-989d-976b4c56beb6 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:18.331643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2406c2af-377f-4bd6-989d-976b4c56beb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:18.438720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c2af-377f-4bd6-989d-976b4c56beb6 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:18.485327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:18.511338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfad0f9-e540-43b9-8f00-4e690193e26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:18.511998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfad0f9-e540-43b9-8f00-4e690193e26e service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:18.512224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfad0f9-e540-43b9-8f00-4e690193e26e service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:18.512493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfad0f9-e540-43b9-8f00-4e690193e26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:18.514134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-abfad0f9-e540-43b9-8f00-4e690193e26e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:18.622888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfad0f9-e540-43b9-8f00-4e690193e26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:18.624330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfad0f9-e540-43b9-8f00-4e690193e26e service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:18.643224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 737/1470] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:28.036953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=22 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:28.640694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=66 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:38.644786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c993a-5bb9-45f9-9c7b-873fffdb11d5 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:38.645269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c993a-5bb9-45f9-9c7b-873fffdb11d5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:38.645364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c993a-5bb9-45f9-9c7b-873fffdb11d5 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:38.646701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 734/1471] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:38.652625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e914e2-6045-40e1-98b1-3dd4558d803a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:38.653085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e914e2-6045-40e1-98b1-3dd4558d803a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:38.653190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e914e2-6045-40e1-98b1-3dd4558d803a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:38.719655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-a9e914e2-6045-40e1-98b1-3dd4558d803a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:38.720885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a9e914e2-6045-40e1-98b1-3dd4558d803a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:38.816694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 738/1472] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 12:59:39.115505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d801f2-45bc-46f4-b23c-56a3ae7e53f2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:39.115783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d801f2-45bc-46f4-b23c-56a3ae7e53f2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:39.116052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d801f2-45bc-46f4-b23c-56a3ae7e53f2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:39.119213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 735/1473] 10.222.0.72 () {54 vars in 966 bytes} [Tue Sep 13 12:59:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:39.124364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89807f58-5eb0-45b3-bd1d-40122cf7f8c7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:39.124649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89807f58-5eb0-45b3-bd1d-40122cf7f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:39.124919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89807f58-5eb0-45b3-bd1d-40122cf7f8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:39.200117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-89807f58-5eb0-45b3-bd1d-40122cf7f8c7 None None] MFA Rules not processed for user `04084ffe1b0548fea75ea144977e4f15`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:39.201317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-89807f58-5eb0-45b3-bd1d-40122cf7f8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:39.221809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:39.276822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 739/1474] 10.222.0.72 () {58 vars in 1063 bytes} [Tue Sep 13 12:59:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:39.284824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d801f2-45bc-46f4-b23c-56a3ae7e53f2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:39.286397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-88d801f2-45bc-46f4-b23c-56a3ae7e53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:39.389448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d801f2-45bc-46f4-b23c-56a3ae7e53f2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:39.463056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219b1d2-c147-4b9e-950b-12f932943fbe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04084ffe1b0548fea75ea144977e4f15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:39.463662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b1d2-c147-4b9e-950b-12f932943fbe service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:39.463941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b1d2-c147-4b9e-950b-12f932943fbe service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:39.464211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219b1d2-c147-4b9e-950b-12f932943fbe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:39.465841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1219b1d2-c147-4b9e-950b-12f932943fbe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:39.519638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:39.575605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219b1d2-c147-4b9e-950b-12f932943fbe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:39.577009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219b1d2-c147-4b9e-950b-12f932943fbe service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:39.594729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 736/1475] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:41.849869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4333e9d-56c2-4174-882f-afc0def7981b None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:41.850645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4333e9d-56c2-4174-882f-afc0def7981b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:41.850645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4333e9d-56c2-4174-882f-afc0def7981b None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:41.851915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 740/1476] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:41.857506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d43a-560c-4e24-bbbc-28931c8881b8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:41.857975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d43a-560c-4e24-bbbc-28931c8881b8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:41.858087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d43a-560c-4e24-bbbc-28931c8881b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:41.923415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c223d43a-560c-4e24-bbbc-28931c8881b8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:41.924553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c223d43a-560c-4e24-bbbc-28931c8881b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:42.021057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 737/1477] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:42.299058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4333e9d-56c2-4174-882f-afc0def7981b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:42.300610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a4333e9d-56c2-4174-882f-afc0def7981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:42.426380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4333e9d-56c2-4174-882f-afc0def7981b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:42.476868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:42.519881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f549-3c30-40c3-90dd-79da99d04f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:42.520490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f549-3c30-40c3-90dd-79da99d04f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:42.520768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f549-3c30-40c3-90dd-79da99d04f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:42.521040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f549-3c30-40c3-90dd-79da99d04f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:42.521676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f549-3c30-40c3-90dd-79da99d04f87 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:42.523987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-fd92f549-3c30-40c3-90dd-79da99d04f87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 12:59:42.523987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 12:59:42.523987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 12:59:42.525134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f549-3c30-40c3-90dd-79da99d04f87 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:42.544351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 741/1478] 10.222.0.72 () {56 vars in 1108 bytes} [Tue Sep 13 12:59:42 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 12:59:44.701552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502799c-4ac2-45c8-a45f-7738233ee6f2 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:44.702199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502799c-4ac2-45c8-a45f-7738233ee6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:44.702199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502799c-4ac2-45c8-a45f-7738233ee6f2 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:44.703454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 738/1479] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:44.708870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967c08-378c-4c0e-8d0f-a9860732b935 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:44.709286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967c08-378c-4c0e-8d0f-a9860732b935 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:44.709413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967c08-378c-4c0e-8d0f-a9860732b935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:44.775091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-97967c08-378c-4c0e-8d0f-a9860732b935 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:44.776306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-97967c08-378c-4c0e-8d0f-a9860732b935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:44.870290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 742/1480] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:45.168974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502799c-4ac2-45c8-a45f-7738233ee6f2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:45.170523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b502799c-4ac2-45c8-a45f-7738233ee6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:45.181529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:45.319870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502799c-4ac2-45c8-a45f-7738233ee6f2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:45.400109 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:45.432557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490b528-0517-4e28-a7a7-ecbb5afeaa97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:45.433098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490b528-0517-4e28-a7a7-ecbb5afeaa97 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:45.433423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490b528-0517-4e28-a7a7-ecbb5afeaa97 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:45.433712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490b528-0517-4e28-a7a7-ecbb5afeaa97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:45.435982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a490b528-0517-4e28-a7a7-ecbb5afeaa97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:45.526111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490b528-0517-4e28-a7a7-ecbb5afeaa97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:45.527694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490b528-0517-4e28-a7a7-ecbb5afeaa97 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:45.545886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 739/1481] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:47.886364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aa59c-ef3b-4075-9bc3-d4a3e11a3ae7 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:47.886883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aa59c-ef3b-4075-9bc3-d4a3e11a3ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:47.886883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977aa59c-ef3b-4075-9bc3-d4a3e11a3ae7 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:47.888173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 743/1482] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:47.893470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573ab9-a1ea-44e3-9410-950434113e81 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:47.894872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573ab9-a1ea-44e3-9410-950434113e81 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:47.894872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573ab9-a1ea-44e3-9410-950434113e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:47.960778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dd573ab9-a1ea-44e3-9410-950434113e81 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:47.961791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dd573ab9-a1ea-44e3-9410-950434113e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:48.008276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:48.059142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 740/1483] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:48.352330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977aa59c-ef3b-4075-9bc3-d4a3e11a3ae7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:48.353922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-977aa59c-ef3b-4075-9bc3-d4a3e11a3ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:48.377966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:48.457588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977aa59c-ef3b-4075-9bc3-d4a3e11a3ae7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:48.531921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a52eb1-5d65-48e3-ba55-2b9d05aa0960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:48.532452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52eb1-5d65-48e3-ba55-2b9d05aa0960 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:48.532734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52eb1-5d65-48e3-ba55-2b9d05aa0960 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:48.533008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52eb1-5d65-48e3-ba55-2b9d05aa0960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:48.534654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d2a52eb1-5d65-48e3-ba55-2b9d05aa0960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:48.610691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:48.649633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a52eb1-5d65-48e3-ba55-2b9d05aa0960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:48.651409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a52eb1-5d65-48e3-ba55-2b9d05aa0960 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:48.671591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 744/1484] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:50.880516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24650096-955c-4310-ab09-c435d8ac89c7 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:50.881076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24650096-955c-4310-ab09-c435d8ac89c7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:50.881076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24650096-955c-4310-ab09-c435d8ac89c7 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:50.883068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 741/1485] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:50.888332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bf6e-272b-4de5-bcd2-be8cfd281340 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:50.888753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bf6e-272b-4de5-bcd2-be8cfd281340 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:50.888866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bf6e-272b-4de5-bcd2-be8cfd281340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:50.954004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6c64bf6e-272b-4de5-bcd2-be8cfd281340 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:50.955203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6c64bf6e-272b-4de5-bcd2-be8cfd281340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:51.049063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 745/1486] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:51.340786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24650096-955c-4310-ab09-c435d8ac89c7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:51.342362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-24650096-955c-4310-ab09-c435d8ac89c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:51.444918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24650096-955c-4310-ab09-c435d8ac89c7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:51.519993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:51.521503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad87516-172d-4717-be61-703c9548a74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:51.522164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad87516-172d-4717-be61-703c9548a74e service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:51.522445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad87516-172d-4717-be61-703c9548a74e service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:51.523312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad87516-172d-4717-be61-703c9548a74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:51.524857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2ad87516-172d-4717-be61-703c9548a74e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:51.635223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad87516-172d-4717-be61-703c9548a74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:51.636581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad87516-172d-4717-be61-703c9548a74e service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:51.655336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 742/1487] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:54.012829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c1b95-8090-4360-86e5-f25c1ab8a109 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:54.013332 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c1b95-8090-4360-86e5-f25c1ab8a109 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:54.013442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c1b95-8090-4360-86e5-f25c1ab8a109 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:54.014759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 746/1488] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 12:59:54.020453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d89d1d-89e7-4d67-bc2e-37e8b62dfd4e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:54.020889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d89d1d-89e7-4d67-bc2e-37e8b62dfd4e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:54.020989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d89d1d-89e7-4d67-bc2e-37e8b62dfd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:54.087877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-24d89d1d-89e7-4d67-bc2e-37e8b62dfd4e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:54.089077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-24d89d1d-89e7-4d67-bc2e-37e8b62dfd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:54.140681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:54.184454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 743/1489] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:54.476027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c1b95-8090-4360-86e5-f25c1ab8a109 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:54.477565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d07c1b95-8090-4360-86e5-f25c1ab8a109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:54.513958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:54.580418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c1b95-8090-4360-86e5-f25c1ab8a109 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:54.653429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4912934-803f-4dc0-ac7c-43e890993e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:54.654065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4912934-803f-4dc0-ac7c-43e890993e6c service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:54.654345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4912934-803f-4dc0-ac7c-43e890993e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:54.654616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4912934-803f-4dc0-ac7c-43e890993e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:54.656207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a4912934-803f-4dc0-ac7c-43e890993e6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:54.736730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:54.765425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4912934-803f-4dc0-ac7c-43e890993e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:54.766957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4912934-803f-4dc0-ac7c-43e890993e6c service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:54.785319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 747/1490] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 12:59:57.284920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7132d-8c43-4c58-9720-7a0053355deb None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:57.285493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7132d-8c43-4c58-9720-7a0053355deb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:57.285493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7132d-8c43-4c58-9720-7a0053355deb None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:57.286834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 744/1491] 10.222.0.72 () {54 vars in 927 bytes} [Tue Sep 13 12:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 12:59:57.292706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92445ac-2488-4a27-ab6f-20699cb9e2b9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:57.293121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92445ac-2488-4a27-ab6f-20699cb9e2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:57.293304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92445ac-2488-4a27-ab6f-20699cb9e2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:57.359716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f92445ac-2488-4a27-ab6f-20699cb9e2b9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 12:59:57.360944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f92445ac-2488-4a27-ab6f-20699cb9e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:57.461142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 748/1492] 10.222.0.72 () {58 vars in 1024 bytes} [Tue Sep 13 12:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 12:59:57.762527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7132d-8c43-4c58-9720-7a0053355deb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 12:59:57.764066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8dc7132d-8c43-4c58-9720-7a0053355deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:57.866049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7132d-8c43-4c58-9720-7a0053355deb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 12:59:57.941732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba5039-cd7e-4e9e-89e5-a2c14e65bf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 12:59:57.942281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5039-cd7e-4e9e-89e5-a2c14e65bf86 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 12:59:57.942560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5039-cd7e-4e9e-89e5-a2c14e65bf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 12:59:57.942832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba5039-cd7e-4e9e-89e5-a2c14e65bf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 12:59:57.944466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-80ba5039-cd7e-4e9e-89e5-a2c14e65bf86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 12:59:57.953536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 12:59:58.052003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba5039-cd7e-4e9e-89e5-a2c14e65bf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 12:59:58.053673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba5039-cd7e-4e9e-89e5-a2c14e65bf86 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 12:59:58.072017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 745/1493] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 12:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:00:07.459072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=82 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:00:08.070115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=50 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:27.217769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:27.218363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:27.218363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:27.294430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:01:27.295672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:27.391563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 749/1494] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:01:27.399591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f121680-188a-4c97-a178-c5355d3c3de8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:27.400106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f121680-188a-4c97-a178-c5355d3c3de8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:27.400215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f121680-188a-4c97-a178-c5355d3c3de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:27.475922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6f121680-188a-4c97-a178-c5355d3c3de8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:01:27.477146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6f121680-188a-4c97-a178-c5355d3c3de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:27.571475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 746/1495] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:01:27.576047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:27.577243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:27.589104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a24f0-2f98-40a8-9020-bf49d0037cd8 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:27.654673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:27.689638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:27.690278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:27.690589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:27.691062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:27.691868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:01:27.692463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:27.693511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:27.693511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 13:01:27.693511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 13:01:27.694445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:27.705946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 750/1496] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:01:27.715004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f121680-188a-4c97-a178-c5355d3c3de8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:27.724470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f121680-188a-4c97-a178-c5355d3c3de8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:27.784777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:27.819224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:27.819839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:27.820110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:27.820379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:27.820998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:27.822751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:27.822751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 13 13:01:27.822751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: Sep 13 13:01:27.823965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:27.971332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 747/1497] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:01:27 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:01:27.975560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:27.978231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:28.129881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaaaf3-6531-48f0-9166-3273d82844a8 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:28.220863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:28.221526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:28.221709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:28.222006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:28.222632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:28.223635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:28.223635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 13:01:28.223635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 13:01:28.224818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:28.364041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 751/1498] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:01:27 2022] POST /identity/v3/users => generated 290 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:01:28.368347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:28.369941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:28.431984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114d44-9467-40b8-a609-80fb0de1a706 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:28.513848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:28.524632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:28.527381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:28.527381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:28.527381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:28.527381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:28.527753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:28.527753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 13 13:01:28.527753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: Sep 13 13:01:28.528353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:28.542953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 748/1499] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:01:28.547103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:28.548629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:28.560501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12680bcd-f1e7-4159-bb19-e8b2a3b13c36 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:28.586952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:28.654613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932c20e-d175-4e78-9af0-2d4936ed4793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:28.655246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c20e-d175-4e78-9af0-2d4936ed4793 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:28.655524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c20e-d175-4e78-9af0-2d4936ed4793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:28.655791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932c20e-d175-4e78-9af0-2d4936ed4793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:28.656426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932c20e-d175-4e78-9af0-2d4936ed4793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:28.657428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-a932c20e-d175-4e78-9af0-2d4936ed4793 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:28.657428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 13:01:28.657428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 13:01:28.658337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932c20e-d175-4e78-9af0-2d4936ed4793 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:28.672070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 752/1500] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:01:28.676503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:28.678040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:28.711386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851b223-b59f-4a80-b4e2-32192cb99dbc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:28.802948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179347e9-5456-439d-a25d-cd93bc7ceffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:28.803596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179347e9-5456-439d-a25d-cd93bc7ceffd admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:28.803872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179347e9-5456-439d-a25d-cd93bc7ceffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:28.804141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179347e9-5456-439d-a25d-cd93bc7ceffd admin admin] PATH_INFO: `/v3/projects/35648d995cd54c6cbe67596e141c7bb6/users/047f1b4aebf640578b9ef7dd673ab9ce/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:28.828940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179347e9-5456-439d-a25d-cd93bc7ceffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=35648d995cd54c6cbe67596e141c7bb6, user_id=047f1b4aebf640578b9ef7dd673ab9ce, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:28.829986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-179347e9-5456-439d-a25d-cd93bc7ceffd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:28.829986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 13 13:01:28.829986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: Sep 13 13:01:28.831887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179347e9-5456-439d-a25d-cd93bc7ceffd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:28.915690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 749/1501] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:01:28 2022] PUT /identity/v3/projects/35648d995cd54c6cbe67596e141c7bb6/users/047f1b4aebf640578b9ef7dd673ab9ce/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:01:28.928245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3cab4-394d-4bcf-8fec-323de55b9d45 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:28.932220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3cab4-394d-4bcf-8fec-323de55b9d45 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:28.932220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3cab4-394d-4bcf-8fec-323de55b9d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:28.984122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-87a3cab4-394d-4bcf-8fec-323de55b9d45 None None] MFA Rules not processed for user `047f1b4aebf640578b9ef7dd673ab9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:01:28.985257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-87a3cab4-394d-4bcf-8fec-323de55b9d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:29.062459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 753/1502] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:01:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:01:29.075660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39a417-252e-44af-b998-3f3717e94070 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:29.075947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39a417-252e-44af-b998-3f3717e94070 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:29.076221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39a417-252e-44af-b998-3f3717e94070 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:29.077563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 750/1503] 10.222.0.72 () {54 vars in 964 bytes} [Tue Sep 13 13:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:01:29.082230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9de6b0-df6f-449c-b691-eb8df8ad8752 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:29.082531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9de6b0-df6f-449c-b691-eb8df8ad8752 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:29.082820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9de6b0-df6f-449c-b691-eb8df8ad8752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:29.115312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:29.160266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-cd9de6b0-df6f-449c-b691-eb8df8ad8752 None None] MFA Rules not processed for user `6c5b4fdb65514ca6b8033f0f1e4c5eab`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:01:29.161553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cd9de6b0-df6f-449c-b691-eb8df8ad8752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:29.235986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 754/1504] 10.222.0.72 () {58 vars in 1061 bytes} [Tue Sep 13 13:01:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:01:29.243725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39a417-252e-44af-b998-3f3717e94070 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:29.245216 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ed39a417-252e-44af-b998-3f3717e94070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:29.299277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:29.352375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39a417-252e-44af-b998-3f3717e94070 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:29.425454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:29.426149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:29.426678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:29.426948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:29.428531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:29.526805 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:29.528116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:29.531570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:29.545355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 751/1505] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:01:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:01:29.591799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e22dc-0483-4117-a502-9e27059c3b79 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:29.592078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e22dc-0483-4117-a502-9e27059c3b79 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:29.592343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e22dc-0483-4117-a502-9e27059c3b79 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:29.593585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 755/1506] 10.222.0.72 () {52 vars in 830 bytes} [Tue Sep 13 13:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:01:29.597156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:29.598399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:29.701514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd0ba3-6a8b-4b41-8142-c35d76989b43 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:29.792376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:29.792925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:29.793200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:29.793468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] PATH_INFO: `/v3/users/047f1b4aebf640578b9ef7dd673ab9ce` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:29.807050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=047f1b4aebf640578b9ef7dd673ab9ce)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:29.808106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:29.808106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 13 13:01:29.808106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: Sep 13 13:01:29.815294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:29.818892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:30.046684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047f1b4aebf640578b9ef7dd673ab9ce'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:30.047063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047f1b4aebf640578b9ef7dd673ab9ce'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:30.063284 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047f1b4aebf640578b9ef7dd673ab9ce'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:30.086736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047f1b4aebf640578b9ef7dd673ab9ce'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:30.087304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047f1b4aebf640578b9ef7dd673ab9ce'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:30.191237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 752/1507] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:01:29 2022] DELETE /identity/v3/users/047f1b4aebf640578b9ef7dd673ab9ce => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:01:30.195699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:30.195983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:30.196253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:30.254598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:01:30.256153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:30.339551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:30.350928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 756/1508] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:01:30.368599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:30.369856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:30.469475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4622409-7e7f-43e4-b753-089649a512ca admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:30.522400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:30.541847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52af15-b3aa-486f-afaa-e2289e8b4ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:30.542474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52af15-b3aa-486f-afaa-e2289e8b4ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:30.542728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52af15-b3aa-486f-afaa-e2289e8b4ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:30.542996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52af15-b3aa-486f-afaa-e2289e8b4ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:30.544652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fb52af15-b3aa-486f-afaa-e2289e8b4ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:30.653470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52af15-b3aa-486f-afaa-e2289e8b4ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:30.654972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52af15-b3aa-486f-afaa-e2289e8b4ab5 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:30.673923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 753/1509] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:01:31.128393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:01:31.129376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:01:31.140993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed410764-de3c-4a9a-964f-834b8d805b57 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:01:31.240727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:01:31.241355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:01:31.241720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:01:31.242325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] PATH_INFO: `/v3/projects/35648d995cd54c6cbe67596e141c7bb6` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:01:31.251814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35648d995cd54c6cbe67596e141c7bb6)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:01:31.252831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:01:31.252831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 13:01:31.252831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 13:01:31.254046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:01:31.376372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35648d995cd54c6cbe67596e141c7bb6', 'request_id': 'req-7df55ca9-2fa0-46d5-b202-81c160aa4737'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:31.431441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35648d995cd54c6cbe67596e141c7bb6', 'request_id': 'req-7df55ca9-2fa0-46d5-b202-81c160aa4737'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:31.446732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35648d995cd54c6cbe67596e141c7bb6', 'request_id': 'req-7df55ca9-2fa0-46d5-b202-81c160aa4737'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:31.469326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] The token cache is being invalidate because project 35648d995cd54c6cbe67596e141c7bb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:01:31.469614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:31.470161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7df55ca9-2fa0-46d5-b202-81c160aa4737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:01:31.471452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 757/1510] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:01:31 2022] DELETE /identity/v3/projects/35648d995cd54c6cbe67596e141c7bb6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:01:40.671670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=62 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:01:41.467392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:02:54.654371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893cdd-ac2f-4179-977b-7fd1fcd763b0 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:54.654999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893cdd-ac2f-4179-977b-7fd1fcd763b0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:54.654999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04893cdd-ac2f-4179-977b-7fd1fcd763b0 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:54.656256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 13:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:02:54.661190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abbe00-dedc-4308-9a1c-f6bfe281a998 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:54.661594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abbe00-dedc-4308-9a1c-f6bfe281a998 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:54.661727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abbe00-dedc-4308-9a1c-f6bfe281a998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:54.740889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e0abbe00-dedc-4308-9a1c-f6bfe281a998 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:02:54.742126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e0abbe00-dedc-4308-9a1c-f6bfe281a998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:02:54.844968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 13:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:02:54.999126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04893cdd-ac2f-4179-977b-7fd1fcd763b0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:02:55.001224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-04893cdd-ac2f-4179-977b-7fd1fcd763b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:02:55.118079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04893cdd-ac2f-4179-977b-7fd1fcd763b0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:02:55.123528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2ab7c-2946-4c0a-84f6-f638f29bebde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:02:55.124130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2ab7c-2946-4c0a-84f6-f638f29bebde service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:55.124411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2ab7c-2946-4c0a-84f6-f638f29bebde service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:55.124684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2ab7c-2946-4c0a-84f6-f638f29bebde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:55.132644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2ab7c-2946-4c0a-84f6-f638f29bebde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:02:55.134347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2ab7c-2946-4c0a-84f6-f638f29bebde service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:02:55.143835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 755/1513] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:02:56.229635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96216c-5ff6-4edd-b9aa-47b2e17f238e None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:56.230211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96216c-5ff6-4edd-b9aa-47b2e17f238e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:56.230322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96216c-5ff6-4edd-b9aa-47b2e17f238e None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:56.231585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 13:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:02:56.238360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c4006-6518-41ad-809d-2fe7973733d3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:56.238849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c4006-6518-41ad-809d-2fe7973733d3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:56.239090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c4006-6518-41ad-809d-2fe7973733d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:56.259244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-141c4006-6518-41ad-809d-2fe7973733d3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:02:56.260455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-141c4006-6518-41ad-809d-2fe7973733d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:02:56.268436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 13:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:02:56.422370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96216c-5ff6-4edd-b9aa-47b2e17f238e None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:02:56.437219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96216c-5ff6-4edd-b9aa-47b2e17f238e None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:02:56.442524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730b58d-759e-44ac-afd5-43ad00e46bc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:02:56.443111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730b58d-759e-44ac-afd5-43ad00e46bc2 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:56.443473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730b58d-759e-44ac-afd5-43ad00e46bc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:56.443729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730b58d-759e-44ac-afd5-43ad00e46bc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:56.451739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730b58d-759e-44ac-afd5-43ad00e46bc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:02:56.453310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730b58d-759e-44ac-afd5-43ad00e46bc2 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:02:56.462444 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 760/1516] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:02:56.768095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba176-6b50-4e2f-92a2-a6adf5166786 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:56.768382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba176-6b50-4e2f-92a2-a6adf5166786 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:56.768660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ba176-6b50-4e2f-92a2-a6adf5166786 None None] PATH_INFO: `/v3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:56.770018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 757/1517] 10.222.0.61 () {54 vars in 925 bytes} [Tue Sep 13 13:02:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:02:56.776361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92ecc-5bb7-4655-8353-9484c3d49a5f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:02:56.776647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92ecc-5bb7-4655-8353-9484c3d49a5f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:02:56.776923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a92ecc-5bb7-4655-8353-9484c3d49a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:02:56.836464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-40a92ecc-5bb7-4655-8353-9484c3d49a5f None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:02:56.837725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-40a92ecc-5bb7-4655-8353-9484c3d49a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:02:56.854599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:02:56.908334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 761/1518] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Sep 13 13:02:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:03:00.696964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349342a-4ff2-4c4b-9594-6864eebdeafb None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:00.697408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349342a-4ff2-4c4b-9594-6864eebdeafb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:00.697505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349342a-4ff2-4c4b-9594-6864eebdeafb None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:00.698874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 758/1519] 10.222.0.72 () {54 vars in 961 bytes} [Tue Sep 13 13:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:03:00.703825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ddf7-90d9-43c0-8588-cad9f788f63b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:00.704215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ddf7-90d9-43c0-8588-cad9f788f63b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:00.704371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ddf7-90d9-43c0-8588-cad9f788f63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:00.723974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-8693ddf7-90d9-43c0-8588-cad9f788f63b None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:03:00.725159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8693ddf7-90d9-43c0-8588-cad9f788f63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:00.733973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 762/1520] 10.222.0.72 () {58 vars in 1058 bytes} [Tue Sep 13 13:03:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:03:00.741236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349342a-4ff2-4c4b-9594-6864eebdeafb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:03:00.754715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349342a-4ff2-4c4b-9594-6864eebdeafb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:03:00.760155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d9aad-45ce-4ff6-8b10-5f389975f1c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:03:00.760746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d9aad-45ce-4ff6-8b10-5f389975f1c7 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:00.761025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d9aad-45ce-4ff6-8b10-5f389975f1c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:00.761300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d9aad-45ce-4ff6-8b10-5f389975f1c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:00.769389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d9aad-45ce-4ff6-8b10-5f389975f1c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:03:00.770986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d9aad-45ce-4ff6-8b10-5f389975f1c7 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:03:00.781132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 759/1521] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:03:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:03:07.952315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99b43a-8b13-474a-a3dd-f6401c4d969e None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:07.952861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99b43a-8b13-474a-a3dd-f6401c4d969e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:07.952974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99b43a-8b13-474a-a3dd-f6401c4d969e None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:07.954324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 13:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:03:07.960414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44daf5-9404-4b80-8f53-739cf5ff7ca6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:07.961367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44daf5-9404-4b80-8f53-739cf5ff7ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:07.961622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44daf5-9404-4b80-8f53-739cf5ff7ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:07.981650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-be44daf5-9404-4b80-8f53-739cf5ff7ca6 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:03:07.982872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-be44daf5-9404-4b80-8f53-739cf5ff7ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:07.990711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 13:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:03:08.144045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99b43a-8b13-474a-a3dd-f6401c4d969e None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:03:08.159074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99b43a-8b13-474a-a3dd-f6401c4d969e None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:03:08.164712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aeaa24-43fe-4652-a72b-bf122dbcf819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:03:08.165302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeaa24-43fe-4652-a72b-bf122dbcf819 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:08.165581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeaa24-43fe-4652-a72b-bf122dbcf819 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:08.165894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeaa24-43fe-4652-a72b-bf122dbcf819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:08.174385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aeaa24-43fe-4652-a72b-bf122dbcf819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:03:08.176016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aeaa24-43fe-4652-a72b-bf122dbcf819 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:03:08.187410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 764/1524] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:03:09.263002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b58db-6f85-44f2-86dc-3c6453231a8d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:09.263569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b58db-6f85-44f2-86dc-3c6453231a8d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:09.263569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b58db-6f85-44f2-86dc-3c6453231a8d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:09.264912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 13:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:03:09.271147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba951f-cdfc-4ebc-a865-3286b264eb6b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:09.271565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba951f-cdfc-4ebc-a865-3286b264eb6b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:09.271699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba951f-cdfc-4ebc-a865-3286b264eb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:09.293021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-aaba951f-cdfc-4ebc-a865-3286b264eb6b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:03:09.294265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-aaba951f-cdfc-4ebc-a865-3286b264eb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:09.302284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 13:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:03:09.454622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b58db-6f85-44f2-86dc-3c6453231a8d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:03:09.469491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b58db-6f85-44f2-86dc-3c6453231a8d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:03:09.474884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105deb31-0d56-41df-b155-306042592a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:03:09.475472 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105deb31-0d56-41df-b155-306042592a7a service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:09.475750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105deb31-0d56-41df-b155-306042592a7a service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:09.476107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105deb31-0d56-41df-b155-306042592a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:09.485385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105deb31-0d56-41df-b155-306042592a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:03:09.486991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105deb31-0d56-41df-b155-306042592a7a service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:03:09.496608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 762/1527] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:03:13.600150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5cf65-b287-4228-9321-c9613b0bfeaf None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:13.600690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5cf65-b287-4228-9321-c9613b0bfeaf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:13.600690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5cf65-b287-4228-9321-c9613b0bfeaf None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:13.602597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 13:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:03:13.607906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0370f18-30fd-4aa4-a3f0-483da6e7b5a8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:13.608338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0370f18-30fd-4aa4-a3f0-483da6e7b5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:13.608521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0370f18-30fd-4aa4-a3f0-483da6e7b5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:13.628504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b0370f18-30fd-4aa4-a3f0-483da6e7b5a8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:03:13.631609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b0370f18-30fd-4aa4-a3f0-483da6e7b5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:13.638105 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 13:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:03:13.927006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5cf65-b287-4228-9321-c9613b0bfeaf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:03:13.928695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-41c5cf65-b287-4228-9321-c9613b0bfeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:14.005336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5cf65-b287-4228-9321-c9613b0bfeaf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:03:14.009109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a4cb1-817b-488b-a6ba-a2377880fada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:03:14.009706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a4cb1-817b-488b-a6ba-a2377880fada service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:14.010016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a4cb1-817b-488b-a6ba-a2377880fada service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:14.010290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a4cb1-817b-488b-a6ba-a2377880fada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:14.019027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a4cb1-817b-488b-a6ba-a2377880fada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:03:14.020449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a4cb1-817b-488b-a6ba-a2377880fada service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:03:14.030254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 767/1530] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:03:16.177660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31043-4127-4b77-984e-55554441dbff None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:16.178328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31043-4127-4b77-984e-55554441dbff None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:16.178328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad31043-4127-4b77-984e-55554441dbff None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:16.179683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 13:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:03:16.184846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0664-1cb1-497e-b339-1075261719f4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:16.185228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0664-1cb1-497e-b339-1075261719f4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:16.185399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0664-1cb1-497e-b339-1075261719f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:16.204522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5b7b0664-1cb1-497e-b339-1075261719f4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:03:16.205718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5b7b0664-1cb1-497e-b339-1075261719f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:16.214053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:03:16.514989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad31043-4127-4b77-984e-55554441dbff None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:03:16.516692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fad31043-4127-4b77-984e-55554441dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:16.530243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad31043-4127-4b77-984e-55554441dbff None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:03:16.534043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204047b-bb0c-415d-98e1-cffa17f4c6c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:03:16.535211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204047b-bb0c-415d-98e1-cffa17f4c6c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:16.535494 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204047b-bb0c-415d-98e1-cffa17f4c6c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:16.535768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204047b-bb0c-415d-98e1-cffa17f4c6c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:16.544897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204047b-bb0c-415d-98e1-cffa17f4c6c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:03:16.546352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204047b-bb0c-415d-98e1-cffa17f4c6c5 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:03:16.556897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 765/1533] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:03:20.402702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1601-27ec-446b-9a4e-c57625c42204 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:20.403162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1601-27ec-446b-9a4e-c57625c42204 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:20.403269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1601-27ec-446b-9a4e-c57625c42204 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:20.404623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 13:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:03:20.409816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd41ad-79af-4303-a77b-b97b8e57ecec None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:20.410251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd41ad-79af-4303-a77b-b97b8e57ecec None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:20.410376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd41ad-79af-4303-a77b-b97b8e57ecec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:20.430767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fbdd41ad-79af-4303-a77b-b97b8e57ecec None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:03:20.431960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fbdd41ad-79af-4303-a77b-b97b8e57ecec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:20.439650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 13:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:03:20.743350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede1601-27ec-446b-9a4e-c57625c42204 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:03:20.745013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8ede1601-27ec-446b-9a4e-c57625c42204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:20.809595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:03:20.823516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede1601-27ec-446b-9a4e-c57625c42204 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:03:20.829044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5038f-6144-4e6e-9550-3f426e808116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:03:20.829044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5038f-6144-4e6e-9550-3f426e808116 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:20.829044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5038f-6144-4e6e-9550-3f426e808116 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:20.829044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5038f-6144-4e6e-9550-3f426e808116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:20.837289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5038f-6144-4e6e-9550-3f426e808116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:03:20.838846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5038f-6144-4e6e-9550-3f426e808116 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:03:20.848129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 770/1536] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:03:23.032293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be28df-f98d-4ff4-958c-7bbdc9400b2e None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:23.032869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be28df-f98d-4ff4-958c-7bbdc9400b2e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:23.032869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be28df-f98d-4ff4-958c-7bbdc9400b2e None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:23.034277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 13:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:03:23.040159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b4d61-6496-4b87-8833-09e3b3430810 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:23.040556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b4d61-6496-4b87-8833-09e3b3430810 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:23.040732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b4d61-6496-4b87-8833-09e3b3430810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:23.060129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-bb2b4d61-6496-4b87-8833-09e3b3430810 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:03:23.061328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bb2b4d61-6496-4b87-8833-09e3b3430810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:23.068955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 13:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:03:23.388358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be28df-f98d-4ff4-958c-7bbdc9400b2e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:03:23.390083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b5be28df-f98d-4ff4-958c-7bbdc9400b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:03:23.398299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:03:23.403965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be28df-f98d-4ff4-958c-7bbdc9400b2e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:03:23.407584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2556f-ee64-4536-a0bb-ae43f0a7ef21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:03:23.408179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2556f-ee64-4536-a0bb-ae43f0a7ef21 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:03:23.408455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2556f-ee64-4536-a0bb-ae43f0a7ef21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:03:23.408724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2556f-ee64-4536-a0bb-ae43f0a7ef21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:03:23.417536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2556f-ee64-4536-a0bb-ae43f0a7ef21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:03:23.418970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2556f-ee64-4536-a0bb-ae43f0a7ef21 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:03:23.429435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 768/1539] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:03:33.055232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=11 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:03:33.446029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:48.544348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e7c1-4ee5-4c98-b4ba-2d1dd6acdb2c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:48.548241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e7c1-4ee5-4c98-b4ba-2d1dd6acdb2c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:48.548756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e7c1-4ee5-4c98-b4ba-2d1dd6acdb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:48.587080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-fec7e7c1-4ee5-4c98-b4ba-2d1dd6acdb2c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:48.594377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fec7e7c1-4ee5-4c98-b4ba-2d1dd6acdb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:48.601582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 772/1540] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:04:48.611197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad451af-76fd-4324-8420-07a150cb85b7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:48.612212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad451af-76fd-4324-8420-07a150cb85b7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:48.612712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad451af-76fd-4324-8420-07a150cb85b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:48.647322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6ad451af-76fd-4324-8420-07a150cb85b7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:48.650773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ad451af-76fd-4324-8420-07a150cb85b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:48.660636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 769/1541] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:48.665887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e7c1-4ee5-4c98-b4ba-2d1dd6acdb2c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:48.685749 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e7c1-4ee5-4c98-b4ba-2d1dd6acdb2c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:48.694468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:48.695888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:48.696336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:48.696851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:48.697665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:48.698172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:48.699936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:48.718167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 773/1542] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:04:48.726892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad451af-76fd-4324-8420-07a150cb85b7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:48.739695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad451af-76fd-4324-8420-07a150cb85b7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:48.747597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883fe1f-9d76-488e-be1a-bc2dd12cffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:48.748472 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fe1f-9d76-488e-be1a-bc2dd12cffd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:48.748825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fe1f-9d76-488e-be1a-bc2dd12cffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:48.749170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883fe1f-9d76-488e-be1a-bc2dd12cffd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:48.749912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883fe1f-9d76-488e-be1a-bc2dd12cffd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:48.751868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883fe1f-9d76-488e-be1a-bc2dd12cffd6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:48.865094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 770/1543] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:48 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:48.869535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:48.882641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a7b7-5f65-4625-ba51-b7f6c01f4c58 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:48.916893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f6c67-2324-462f-a818-2dab4f09b6f8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:48.917581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f6c67-2324-462f-a818-2dab4f09b6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:48.918177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f6c67-2324-462f-a818-2dab4f09b6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:48.967132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7d3f6c67-2324-462f-a818-2dab4f09b6f8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:48.968795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7d3f6c67-2324-462f-a818-2dab4f09b6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:48.971735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:48.972731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:48.973212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:48.973655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:48.974534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:48.977356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:48.986916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 771/1544] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:04:48.996850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a521-9684-4369-aa20-ec5e221d2519 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:48.997281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a521-9684-4369-aa20-ec5e221d2519 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:48.997656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7a521-9684-4369-aa20-ec5e221d2519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.047621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-79c7a521-9684-4369-aa20-ec5e221d2519 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.049128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-79c7a521-9684-4369-aa20-ec5e221d2519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.063851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 772/1545] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:04:49.068993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a521-9684-4369-aa20-ec5e221d2519 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.080289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7a521-9684-4369-aa20-ec5e221d2519 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.090274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.093113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.093113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.093113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.093113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:49.093113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.093679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:04:49.093679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 13 13:04:49.093679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: Sep 13 13:04:49.094301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 774/1546] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:48 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:49.094606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.100318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.119223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 773/1547] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:04:49.126921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dd56-9231-41de-a2ea-6272c73bbba5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.130657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.136537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.137474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.138021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.138537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.139411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.141248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.142324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54112a3-2014-42a4-9f2e-1d58d39a475d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.158461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86046268-c0fd-4d1c-82e0-6523beffac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.159078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86046268-c0fd-4d1c-82e0-6523beffac65 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.159356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86046268-c0fd-4d1c-82e0-6523beffac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.159624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86046268-c0fd-4d1c-82e0-6523beffac65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.160378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86046268-c0fd-4d1c-82e0-6523beffac65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.162477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86046268-c0fd-4d1c-82e0-6523beffac65 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.162674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 775/1548] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:49.169298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.194427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad9dcb-54b6-475e-b701-1ade473f35dc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.206247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.206885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.207148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.207426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.208101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.209760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.223791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 776/1549] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:04:49.228236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.245664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c378b-3e73-47a4-baa4-065b865ab8bb admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.254296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.255119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.255402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.255674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] PATH_INFO: `/v3/projects/eaaf8aa87a0f47cc9ba9f477de3c2d67/users/e54f17d6995f4c099708d79715d95220/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.276973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaaf8aa87a0f47cc9ba9f477de3c2d67, user_id=e54f17d6995f4c099708d79715d95220, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.278650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:04:49.278650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 13:04:49.278650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 13:04:49.280116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.306844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 774/1550] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:04:49.309669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c9e14-a479-49d6-891d-99b4e6b8a69e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.309979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c9e14-a479-49d6-891d-99b4e6b8a69e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.310251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c9e14-a479-49d6-891d-99b4e6b8a69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.338747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4e3c9e14-a479-49d6-891d-99b4e6b8a69e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.340046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4e3c9e14-a479-49d6-891d-99b4e6b8a69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.359656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 777/1551] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:49 2022] PUT /identity/v3/projects/eaaf8aa87a0f47cc9ba9f477de3c2d67/users/e54f17d6995f4c099708d79715d95220/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:04:49.360246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.438871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 775/1552] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.440037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb6fdf-b597-4b1c-9da7-14b26301c467 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.440321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb6fdf-b597-4b1c-9da7-14b26301c467 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.440590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb6fdf-b597-4b1c-9da7-14b26301c467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.451087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b590635-cfc3-4efa-bbec-e6c7183bad96 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.455742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cf805-8687-46de-81db-5da83cc3fe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.456349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cf805-8687-46de-81db-5da83cc3fe43 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.456632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cf805-8687-46de-81db-5da83cc3fe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.456908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cf805-8687-46de-81db-5da83cc3fe43 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.457543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cf805-8687-46de-81db-5da83cc3fe43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.459441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-aeeb6fdf-b597-4b1c-9da7-14b26301c467 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.459574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cf805-8687-46de-81db-5da83cc3fe43 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.460792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-aeeb6fdf-b597-4b1c-9da7-14b26301c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.468767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 776/1553] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.469965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ae10-9b78-4cd5-a5db-fc0dbc49da3e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.470555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ae10-9b78-4cd5-a5db-fc0dbc49da3e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.470555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ae10-9b78-4cd5-a5db-fc0dbc49da3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.490195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-eaf4ae10-9b78-4cd5-a5db-fc0dbc49da3e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.491417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-eaf4ae10-9b78-4cd5-a5db-fc0dbc49da3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.500027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 777/1554] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.501181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180253c-4b13-4a85-8b41-a96172d3dc4d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.501464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180253c-4b13-4a85-8b41-a96172d3dc4d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.501754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180253c-4b13-4a85-8b41-a96172d3dc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.520270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1180253c-4b13-4a85-8b41-a96172d3dc4d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.523315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1180253c-4b13-4a85-8b41-a96172d3dc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.529549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 778/1555] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.530809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899e97b-5e61-4c64-be53-bf37fe9df378 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.531098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899e97b-5e61-4c64-be53-bf37fe9df378 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.531377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899e97b-5e61-4c64-be53-bf37fe9df378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.551517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4899e97b-5e61-4c64-be53-bf37fe9df378 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.552697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4899e97b-5e61-4c64-be53-bf37fe9df378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.561304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 779/1556] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.562493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd8b6b-9ae2-4855-a1a1-c49b59fbb03d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.562778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd8b6b-9ae2-4855-a1a1-c49b59fbb03d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.563056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd8b6b-9ae2-4855-a1a1-c49b59fbb03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.575732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 778/1557] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/users => generated 282 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:49.580155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2b441-9e92-42b6-ade2-526fe3822e1d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.580155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2b441-9e92-42b6-ade2-526fe3822e1d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.580155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2b441-9e92-42b6-ade2-526fe3822e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.582432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a4dd8b6b-9ae2-4855-a1a1-c49b59fbb03d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.583632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a4dd8b6b-9ae2-4855-a1a1-c49b59fbb03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.592819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 780/1558] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.593960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee80b3-a2ba-49f1-b468-9bdaac2b837b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.594246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee80b3-a2ba-49f1-b468-9bdaac2b837b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.594521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee80b3-a2ba-49f1-b468-9bdaac2b837b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.604147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-b6d2b441-9e92-42b6-ade2-526fe3822e1d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.605374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b6d2b441-9e92-42b6-ade2-526fe3822e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.614908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 779/1559] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.615215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bfee80b3-a2ba-49f1-b468-9bdaac2b837b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.615986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a8d3-d62b-4648-80a0-bf387b669cf7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.616264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a8d3-d62b-4648-80a0-bf387b669cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.617166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bfee80b3-a2ba-49f1-b468-9bdaac2b837b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.617317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94a8d3-d62b-4648-80a0-bf387b669cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.629777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 781/1560] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.629777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092ab8-a1c3-4df1-aba6-92a444619c72 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.629777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092ab8-a1c3-4df1-aba6-92a444619c72 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.629777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092ab8-a1c3-4df1-aba6-92a444619c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.631506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:49.638736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5c94a8d3-d62b-4648-80a0-bf387b669cf7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.639949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5c94a8d3-d62b-4648-80a0-bf387b669cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.652296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 780/1561] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.652488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-83092ab8-a1c3-4df1-aba6-92a444619c72 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.652612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a8d3-d62b-4648-80a0-bf387b669cf7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.652710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-83092ab8-a1c3-4df1-aba6-92a444619c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.659198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 782/1562] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.662047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a6d60-7790-4ab5-9a17-28dd893edcf4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.662047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a6d60-7790-4ab5-9a17-28dd893edcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.662047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a6d60-7790-4ab5-9a17-28dd893edcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.683770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fc0a6d60-7790-4ab5-9a17-28dd893edcf4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.684957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc0a6d60-7790-4ab5-9a17-28dd893edcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.692712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 783/1563] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.694375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a6d60-7790-4ab5-9a17-28dd893edcf4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.704028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a6d60-7790-4ab5-9a17-28dd893edcf4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.708974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b854e4-9910-4ba8-8afa-4b02808d51c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.709290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94a8d3-d62b-4648-80a0-bf387b669cf7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.709569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b854e4-9910-4ba8-8afa-4b02808d51c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.709897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b854e4-9910-4ba8-8afa-4b02808d51c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.710669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b854e4-9910-4ba8-8afa-4b02808d51c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.711310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b854e4-9910-4ba8-8afa-4b02808d51c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.712883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b854e4-9910-4ba8-8afa-4b02808d51c5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.713910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf6262-b2e4-49bf-ae60-c9331f686924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.714551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6262-b2e4-49bf-ae60-c9331f686924 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.714811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6262-b2e4-49bf-ae60-c9331f686924 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.715102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6262-b2e4-49bf-ae60-c9331f686924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.723127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 784/1564] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:49.724306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0656f-2426-42a2-8014-4a908ad1dfe0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.724593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0656f-2426-42a2-8014-4a908ad1dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.724700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf6262-b2e4-49bf-ae60-c9331f686924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.724874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0656f-2426-42a2-8014-4a908ad1dfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.726779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf6262-b2e4-49bf-ae60-c9331f686924 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.736530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 781/1565] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:49.737601 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dedb6-28c7-473e-a8be-075359a980db None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.737922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dedb6-28c7-473e-a8be-075359a980db None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.738194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dedb6-28c7-473e-a8be-075359a980db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.745072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-afa0656f-2426-42a2-8014-4a908ad1dfe0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.746551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-afa0656f-2426-42a2-8014-4a908ad1dfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.758295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 785/1566] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.758295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0656f-2426-42a2-8014-4a908ad1dfe0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.759881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e41dedb6-28c7-473e-a8be-075359a980db None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:49.760987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e41dedb6-28c7-473e-a8be-075359a980db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:49.762136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:49.765693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0656f-2426-42a2-8014-4a908ad1dfe0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.769195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 782/1567] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:49.773309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dedb6-28c7-473e-a8be-075359a980db None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.773440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.773440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.773440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.773440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.773791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:49.773791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.774687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.780382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dedb6-28c7-473e-a8be-075359a980db None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.785073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.785673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.787512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.788174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 786/1568] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:04:49.788174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.794625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.795395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:49.795718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.797597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.801939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d041143-ec0c-48b8-bcca-e88c0d4bab8c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.803745 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.804378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.804669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.804959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.805611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:49.805945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.807530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.810342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 783/1569] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:04:49.814999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.819849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 787/1570] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:04:49.825081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.832729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99783a46-9783-4653-9862-75a4173a2da9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.838585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31b41b-ad16-4952-b33d-595ff3b5f38d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.839392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.840141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.840504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.842458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.843537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:49.844043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.844890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.845578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.845954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.846360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.846743 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.847151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.849014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.862187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 788/1572] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:49.862389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 784/1572] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:04:49.866690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.866864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.878879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae0130-d01d-446e-99a3-8ccd0d401f4d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.880308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e06cf0-1fbe-4027-9e77-ea3382e56365 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.883663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.888129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.888129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.888129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.888506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.888617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:49.888617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.888795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.888795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.888795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.888795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:04:49.889125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.889221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.889314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.900319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 785/1573] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:04:49.901229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.912713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 789/1574] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:04:49.923234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229a9cf-8081-4adc-ba75-f4dd0ee1c94d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.923420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:49.923523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.923523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.923523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.923523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.923862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.923862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:04:49.923862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 13:04:49.923862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 13:04:49.923862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:49.926559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c83ad6-e7ee-4381-88a9-09fc3f46c898 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:49.932492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:49.933107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:49.933388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:49.934102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:49.934357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:49.936272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.041185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 786/1575] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.045199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 790/1576] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:49 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.045963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.046427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.124284 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cad3e-4d0e-4c9d-be13-77c486a48863 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.134329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc253bc-e318-47bd-9e8e-b66517d6d873 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.134462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.134462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.134462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.134462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] PATH_INFO: `/v3/projects/f38ac749002a4725a20b4d0037f33952/users/a2eb560030db496bb256054e6c50a32e/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.134813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.134813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.134813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.134813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.135125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.136028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.147088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38ac749002a4725a20b4d0037f33952, user_id=a2eb560030db496bb256054e6c50a32e, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.151286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.246194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 791/1577] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:50 2022] PUT /identity/v3/projects/f38ac749002a4725a20b4d0037f33952/users/a2eb560030db496bb256054e6c50a32e/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:50.248630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 787/1578] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/projects => generated 391 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.252613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.266106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.280048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:50.330129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe51de1-1b67-49df-9efd-3706e4a2f333 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.331086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22dbe1-d87c-4864-a45f-d5c533b59c89 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.335103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.336270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.336552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.336830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.337463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.337809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.338451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.338787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.339072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.339545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.339756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.341766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.451431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 788/1579] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/projects => generated 355 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.452257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.464473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:50.472793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 792/1580] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/projects => generated 371 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.472793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.523207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80b4ef-2038-41ae-8c7c-eff08a7c7880 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.548137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a336c-24cf-44fa-9aad-d88d3fca190d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.554318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.555149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.555529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.556732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.557478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.558667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:04:50.558667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 13 13:04:50.558667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: Sep 13 13:04:50.561974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.587097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.587801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.588149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.588544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.589255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.591320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.668139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 793/1581] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.669113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.679718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85bcbe-29e6-4948-9ad6-ae022eeff261 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.684962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714858b-0734-4716-b456-b7af9204b23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.685418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 789/1582] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.685567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714858b-0734-4716-b456-b7af9204b23f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.685924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714858b-0734-4716-b456-b7af9204b23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.686200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714858b-0734-4716-b456-b7af9204b23f admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.686307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.686829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714858b-0734-4716-b456-b7af9204b23f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.689095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714858b-0734-4716-b456-b7af9204b23f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.728182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:50.748670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155ad6f-ea8f-44a8-9ce3-6a1c68f3ea11 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.753357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.754007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.754292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.754567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.755206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.757190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.802076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 794/1583] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.802076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564bef-fc9a-4784-8d73-1e0690de40de None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.802076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564bef-fc9a-4784-8d73-1e0690de40de None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.802076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564bef-fc9a-4784-8d73-1e0690de40de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.833256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-06564bef-fc9a-4784-8d73-1e0690de40de None None] MFA Rules not processed for user `a2eb560030db496bb256054e6c50a32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:50.834513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-06564bef-fc9a-4784-8d73-1e0690de40de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:50.867566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 790/1584] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.868281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.877620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c3897-904f-4a4b-90e6-ed31fbb5de21 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.883248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.883844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.884120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.884458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.885055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.886989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.902210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 795/1585] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:50.903182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564bef-fc9a-4784-8d73-1e0690de40de None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.913039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564bef-fc9a-4784-8d73-1e0690de40de None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:50.917994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:50.918596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:50.918877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:50.919152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:50.919781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:50.921963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:50.987647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 791/1586] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/users => generated 278 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:50.989643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:50.999590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83b690-8006-4628-b79b-c2b1246e50ad admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.004507 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.005084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.005361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.005634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.006291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.008781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.019926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 796/1587] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:50 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:51.019926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.021612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 792/1588] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.022684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.034759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936013e-dfb5-4e18-9670-4f63bbd3b46b admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.035415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af8a26-7cc0-488a-8dd4-e2519abe8020 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.039764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.040275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.040418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.040667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.040929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.041339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.041576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.041918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.047814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.047814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.048040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.048180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.054565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 793/1589] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.055740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.066195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30928261-58af-49ff-978c-1d65f1af9737 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.071937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.076481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.076481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.076481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.076481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.076481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.086869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 794/1590] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.090705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.100347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9bf61-8a63-4cf4-b90b-b42a66a2db92 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.106391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.111832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.111832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.111832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.111832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.111832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.145014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 797/1591] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:51 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:51.145014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.156549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d5a9c-e12d-45e9-969a-4a9c5de67fa4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.161223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.161860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.162140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.162416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.163055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.164639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.176926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 798/1592] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.177570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.189977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf54bd-5785-4de2-9666-6bbc30d68f05 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.195383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.195965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.196242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.196513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.197145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.198722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.208776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 799/1593] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.208776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.217864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 795/1594] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:51 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:51.217864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.218640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:51.222555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e6722-7d91-43bf-aa2f-f30a42640737 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.293945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.294653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.294937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.295266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.295980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.298998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.315791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 800/1595] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.316016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481061b6-e488-489d-a94b-e3aa3fb1b8ec admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.316129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.321100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.321772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.322138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.327361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.327361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.327361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.327725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a0e1d-810c-43d3-8563-8c89124a4801 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.332031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.332738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.333125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.333556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.334396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.337872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 796/1596] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.338048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.338151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.348797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f9f3c-6616-4a80-be90-3269e09273a5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.352460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 801/1597] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.356033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.356586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.357204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.357486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.357786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.358440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.360203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.366811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d4c8d-e272-442c-a8f5-3100bdd2c890 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.371543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 797/1598] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.372306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.374691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.377964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.378673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.379110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.380081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.383352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.385174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e753241-7529-436b-952a-60d8b306f619 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.389953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.390564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.390858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.391117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.392121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.393678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.402470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 802/1599] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.403168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.409338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 798/1600] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.412776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.422181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108d6f5-dd65-4b2e-8fa6-aaad4cb229b8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.427141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a409e-237c-4486-b424-6b0488bff2f2 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.428850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.429868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.430408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.430904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.431799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.432587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.433243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.433522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.433824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] PATH_INFO: `/v3/projects/a4e94c8248874defa3cb01d0725dfcb0/users/fe3be97820d64f85aa4d2dbd8f3ffd33/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.434931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.452249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e94c8248874defa3cb01d0725dfcb0, user_id=fe3be97820d64f85aa4d2dbd8f3ffd33, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.455159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.543608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 799/1601] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:51 2022] PUT /identity/v3/projects/a4e94c8248874defa3cb01d0725dfcb0/users/fe3be97820d64f85aa4d2dbd8f3ffd33/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:51.544717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.546054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 803/1602] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:51 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:51.547375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.553891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:51.560262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5bfb5-a68e-434e-a089-f4a0f9b53517 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.626052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52365a91-d6dc-4a25-be8c-4a5a47222d82 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.637290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.637950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.638232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.639152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] PATH_INFO: `/v3/projects/75f6274158934ab5abf68951902f1e81/users/7b1c15ff53e5442881b56bf504b9a31b/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.662782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.663806 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.664279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.664726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] PATH_INFO: `/v3/projects/c15624f4312f49bb85a7a086c291d157/users/8c688fff323943b3bbcbb0569d974d0a/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.665071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f6274158934ab5abf68951902f1e81, user_id=7b1c15ff53e5442881b56bf504b9a31b, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.670107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.682611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15624f4312f49bb85a7a086c291d157, user_id=8c688fff323943b3bbcbb0569d974d0a, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.686093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.754090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 800/1603] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:51 2022] PUT /identity/v3/projects/75f6274158934ab5abf68951902f1e81/users/7b1c15ff53e5442881b56bf504b9a31b/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:51.755384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.783529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 804/1604] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:51 2022] PUT /identity/v3/projects/c15624f4312f49bb85a7a086c291d157/users/8c688fff323943b3bbcbb0569d974d0a/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:51.783529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.833561 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff498f5d-d8c1-4b85-8e67-22bd5947d310 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.847251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:51.867308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798424c-d080-4e40-b521-badbe6da851d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.873026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.873812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.874192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.874651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] PATH_INFO: `/v3/projects/59ba2777b165453ebbafa35810212015/users/0c80935a0ae9472ab49c1e72cac50f25/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.900358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ba2777b165453ebbafa35810212015, user_id=0c80935a0ae9472ab49c1e72cac50f25, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.903662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.912054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.912705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.912985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.913281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.914304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.915955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.927431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 801/1605] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:51.928397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:51.941703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4252d0c-d378-46bd-8ef7-9924199620af admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:51.948507 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eac7ac-efd2-43ea-a3c0-cc75a3290169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:51.948686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac7ac-efd2-43ea-a3c0-cc75a3290169 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:51.948979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac7ac-efd2-43ea-a3c0-cc75a3290169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:51.949766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eac7ac-efd2-43ea-a3c0-cc75a3290169 admin admin] PATH_INFO: `/v3/projects/439cc533cdb04e508aeb56243998d880/users/f76a56d0dbeb43d1b0e2d87b1a27544c/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:51.973353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac7ac-efd2-43ea-a3c0-cc75a3290169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=439cc533cdb04e508aeb56243998d880, user_id=f76a56d0dbeb43d1b0e2d87b1a27544c, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:51.976560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eac7ac-efd2-43ea-a3c0-cc75a3290169 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:51.993469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 805/1606] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:51 2022] PUT /identity/v3/projects/59ba2777b165453ebbafa35810212015/users/0c80935a0ae9472ab49c1e72cac50f25/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:51.995257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.075882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 802/1607] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:51 2022] PUT /identity/v3/projects/439cc533cdb04e508aeb56243998d880/users/f76a56d0dbeb43d1b0e2d87b1a27544c/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:52.075882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f72a04-3702-4be1-9a28-c7a366cfd8e9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.075882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f72a04-3702-4be1-9a28-c7a366cfd8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.075882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f72a04-3702-4be1-9a28-c7a366cfd8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.076650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41943c07-f3cd-455b-84ef-5c67e851c83f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.098930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:52.179369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-70f72a04-3702-4be1-9a28-c7a366cfd8e9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:52.179369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-70f72a04-3702-4be1-9a28-c7a366cfd8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:52.220207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d44d3-1070-4a7d-a5e2-c112af650b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.220880 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d44d3-1070-4a7d-a5e2-c112af650b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.221243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d44d3-1070-4a7d-a5e2-c112af650b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.221583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d44d3-1070-4a7d-a5e2-c112af650b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.222919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d44d3-1070-4a7d-a5e2-c112af650b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.224314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d44d3-1070-4a7d-a5e2-c112af650b2e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.228897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 803/1608] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:52.232596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d964f-20ec-460b-a84b-63268123f34a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.232596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d964f-20ec-460b-a84b-63268123f34a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.232861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d964f-20ec-460b-a84b-63268123f34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.240901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 806/1609] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:52.249693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4e90-b3a2-4da1-87e7-a89c13c4abcf None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.251640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4e90-b3a2-4da1-87e7-a89c13c4abcf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.251980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea4e90-b3a2-4da1-87e7-a89c13c4abcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.265240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e89d964f-20ec-460b-a84b-63268123f34a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:52.266678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e89d964f-20ec-460b-a84b-63268123f34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:52.275426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-37ea4e90-b3a2-4da1-87e7-a89c13c4abcf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:52.276233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 804/1610] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:52.277189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d964f-20ec-460b-a84b-63268123f34a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.277864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-37ea4e90-b3a2-4da1-87e7-a89c13c4abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:52.288084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 807/1611] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:52.290187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f31663-438a-4c14-8c87-3e0889910fc6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.290482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f31663-438a-4c14-8c87-3e0889910fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.291061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f31663-438a-4c14-8c87-3e0889910fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.305009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d964f-20ec-460b-a84b-63268123f34a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.311656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109da8d8-46d6-40d2-80ce-b508726ced9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.312508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da8d8-46d6-40d2-80ce-b508726ced9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.313013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da8d8-46d6-40d2-80ce-b508726ced9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.313381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109da8d8-46d6-40d2-80ce-b508726ced9d admin admin] PATH_INFO: `/v3/projects/7775bd9f19b64c8ca07407c103dcff9b/users/9fadb80f21614816b3526d59b2d8a0f7/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.333224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109da8d8-46d6-40d2-80ce-b508726ced9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7775bd9f19b64c8ca07407c103dcff9b, user_id=9fadb80f21614816b3526d59b2d8a0f7, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.337169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109da8d8-46d6-40d2-80ce-b508726ced9d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.355092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-19f31663-438a-4c14-8c87-3e0889910fc6 None None] MFA Rules not processed for user `0c80935a0ae9472ab49c1e72cac50f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:52.356623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-19f31663-438a-4c14-8c87-3e0889910fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:52.409702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 805/1612] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:52 2022] PUT /identity/v3/projects/7775bd9f19b64c8ca07407c103dcff9b/users/9fadb80f21614816b3526d59b2d8a0f7/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:52.412396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f0dfe-64aa-4768-8d81-410907afea90 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.412974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f0dfe-64aa-4768-8d81-410907afea90 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.414035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f0dfe-64aa-4768-8d81-410907afea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.433051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:52.444439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 808/1613] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:52.448757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f31663-438a-4c14-8c87-3e0889910fc6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.463681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5a0f0dfe-64aa-4768-8d81-410907afea90 None None] MFA Rules not processed for user `f76a56d0dbeb43d1b0e2d87b1a27544c`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:52.465088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5a0f0dfe-64aa-4768-8d81-410907afea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:52.535509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f31663-438a-4c14-8c87-3e0889910fc6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.541619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 806/1614] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:52.542825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f0dfe-64aa-4768-8d81-410907afea90 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.616004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.617627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.617998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.617998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.628258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f0dfe-64aa-4768-8d81-410907afea90 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.631307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.634551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.638207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.638919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.638919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.639199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] PATH_INFO: `/v3/projects/b1635a8e42be4165a09715645943df03/users/2d3af52c82614b5396a14c760c8be6a8/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.658991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 809/1615] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:52.660409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.667411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:52.676368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1635a8e42be4165a09715645943df03, user_id=2d3af52c82614b5396a14c760c8be6a8, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.680579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.701445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7021ea9-43e1-450b-8828-1e7d0cffccc7 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.701445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.701445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.701445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.702459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.711148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.712875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.725036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 810/1616] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:52.726891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.741628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb7dcd-33e9-496d-bf22-e6f8f9747880 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.744560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f262d20-86f4-4b65-83a2-0505730f6410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.745437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f262d20-86f4-4b65-83a2-0505730f6410 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.745811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f262d20-86f4-4b65-83a2-0505730f6410 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.746093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f262d20-86f4-4b65-83a2-0505730f6410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.758800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f262d20-86f4-4b65-83a2-0505730f6410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.764393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f262d20-86f4-4b65-83a2-0505730f6410 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.771125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 811/1617] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:52.772428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8deaa-0b7a-4b30-afb6-8a002b1e598f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.773247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8deaa-0b7a-4b30-afb6-8a002b1e598f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.773544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8deaa-0b7a-4b30-afb6-8a002b1e598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.789237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 807/1618] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:52 2022] PUT /identity/v3/projects/b1635a8e42be4165a09715645943df03/users/2d3af52c82614b5396a14c760c8be6a8/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:52.790160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.800833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0bc8deaa-0b7a-4b30-afb6-8a002b1e598f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:52.802491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0bc8deaa-0b7a-4b30-afb6-8a002b1e598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:52.883060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 812/1619] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:52.885072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b57645-b6ab-4071-b3ca-9a198e89ae51 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.885072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b57645-b6ab-4071-b3ca-9a198e89ae51 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.885481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b57645-b6ab-4071-b3ca-9a198e89ae51 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.893550 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 813/1620] 10.222.0.72 () {54 vars in 962 bytes} [Tue Sep 13 13:04:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:04:52.894807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b57645-b6ab-4071-b3ca-9a198e89ae51 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:52.909423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b57645-b6ab-4071-b3ca-9a198e89ae51 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.916187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c276f6a-41ca-4302-8731-6f5a03c3a3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.916408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe64088-820f-445c-9af3-fbf0ccda3e9a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:52.919578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276f6a-41ca-4302-8731-6f5a03c3a3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.919578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276f6a-41ca-4302-8731-6f5a03c3a3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.919578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276f6a-41ca-4302-8731-6f5a03c3a3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.919578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c276f6a-41ca-4302-8731-6f5a03c3a3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.919975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c276f6a-41ca-4302-8731-6f5a03c3a3a9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.926849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:52.927812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.928486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.930210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.931319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:52.935286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 814/1621] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:52.935592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:52.942747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457842e-52fd-4b43-a8ff-1f9b820df13f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:52.943049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457842e-52fd-4b43-a8ff-1f9b820df13f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:52.943330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457842e-52fd-4b43-a8ff-1f9b820df13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:52.989083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:53.002928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5457842e-52fd-4b43-a8ff-1f9b820df13f None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:53.004754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5457842e-52fd-4b43-a8ff-1f9b820df13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:53.039430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 808/1622] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/projects => generated 357 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:53.040506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.103873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 815/1623] 10.222.0.72 () {58 vars in 1059 bytes} [Tue Sep 13 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:53.104776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457842e-52fd-4b43-a8ff-1f9b820df13f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.139285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457842e-52fd-4b43-a8ff-1f9b820df13f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:53.148000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf32b-7831-4797-8e41-185b9e7fcc23 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:53.151999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:53.154398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:53.154398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:53.154398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:53.232518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:53.234792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:53.245227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:53.245986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:53.246449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:53.247348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 809/1624] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:53.247598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] PATH_INFO: `/v3/projects/b1635a8e42be4165a09715645943df03/users/2d3af52c82614b5396a14c760c8be6a8/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:53.248691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.250473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1635a8e42be4165a09715645943df03, user_id=2d3af52c82614b5396a14c760c8be6a8, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:53.254354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:53.260876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede82a18-48ae-4a1e-acf7-a960edbcf434 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:53.267918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 816/1625] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:53 2022] PUT /identity/v3/projects/b1635a8e42be4165a09715645943df03/users/2d3af52c82614b5396a14c760c8be6a8/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:53.269067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.363788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d729c-5d1f-4a13-9d3f-a3d37c47c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:53.364556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d729c-5d1f-4a13-9d3f-a3d37c47c00b admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:53.364920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d729c-5d1f-4a13-9d3f-a3d37c47c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:53.365778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d729c-5d1f-4a13-9d3f-a3d37c47c00b admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:53.367215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d729c-5d1f-4a13-9d3f-a3d37c47c00b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:53.369718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d729c-5d1f-4a13-9d3f-a3d37c47c00b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:53.412379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5d1a3-20fb-4302-a0b1-54c25e7cdd2d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:53.420101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:53.420885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:53.421312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:53.422150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:53.467972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:53.493462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 810/1626] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:04:53 2022] POST /identity/v3/users => generated 285 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:04:53.493462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad713-cde0-4208-b6ac-a8ca8506e8b2 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:53.493462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad713-cde0-4208-b6ac-a8ca8506e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:53.493462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad713-cde0-4208-b6ac-a8ca8506e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:53.540859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:53.556679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:53.559355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-282ad713-cde0-4208-b6ac-a8ca8506e8b2 None None] MFA Rules not processed for user `2d3af52c82614b5396a14c760c8be6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:53.560630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-282ad713-cde0-4208-b6ac-a8ca8506e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:53.590444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 817/1627] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:53.591630 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.606982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651de6e1-5885-4a6c-a479-26335896d127 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:53.612513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:53.613159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:53.613932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:53.614351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:53.618692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:53.620533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:53.634631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 818/1628] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:04:53.639752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.656797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a161660-819b-428b-aaa8-e11c96f20df4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:53.665412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:53.682392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:53.683087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:53.683371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:53.683653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] PATH_INFO: `/v3/projects/a4ba5a78562b439ebb0667c0f09c2507/users/a0b353f4aae24694afb146e9b681f171/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:53.740844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ba5a78562b439ebb0667c0f09c2507, user_id=a0b353f4aae24694afb146e9b681f171, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:53.748518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:53.759183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 811/1629] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:53.769851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad713-cde0-4208-b6ac-a8ca8506e8b2 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.852536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 819/1630] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:53 2022] PUT /identity/v3/projects/a4ba5a78562b439ebb0667c0f09c2507/users/a0b353f4aae24694afb146e9b681f171/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:53.857219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:53.925173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad713-cde0-4208-b6ac-a8ca8506e8b2 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:53.997894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b3206-1b05-4150-9760-f0c3f4818baf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:54.004705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fe32f-11b3-4790-8074-5810327e10e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:54.007773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe32f-11b3-4790-8074-5810327e10e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.007773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe32f-11b3-4790-8074-5810327e10e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.007773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe32f-11b3-4790-8074-5810327e10e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.009972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ef05a-04c7-46fd-86fb-21e19eeccc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:54.010615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef05a-04c7-46fd-86fb-21e19eeccc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.010899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef05a-04c7-46fd-86fb-21e19eeccc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.011176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ef05a-04c7-46fd-86fb-21e19eeccc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.011833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ef05a-04c7-46fd-86fb-21e19eeccc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:54.013434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ef05a-04c7-46fd-86fb-21e19eeccc76 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:54.041811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 820/1631] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:04:54.043404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72571c93-a73c-4762-883c-1ecd22f27a85 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.370462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72571c93-a73c-4762-883c-1ecd22f27a85 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.370462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72571c93-a73c-4762-883c-1ecd22f27a85 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.370462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 821/1632] 10.222.0.72 () {54 vars in 912 bytes} [Tue Sep 13 13:04:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:04:54.370462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72571c93-a73c-4762-883c-1ecd22f27a85 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:54.370462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72571c93-a73c-4762-883c-1ecd22f27a85 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:54.372728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377789bc-12a3-496d-a29b-16d62f5fd64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:54.372728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377789bc-12a3-496d-a29b-16d62f5fd64c admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.372728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377789bc-12a3-496d-a29b-16d62f5fd64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.372728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377789bc-12a3-496d-a29b-16d62f5fd64c admin admin] PATH_INFO: `/v3/projects/a4ba5a78562b439ebb0667c0f09c2507/users/a0b353f4aae24694afb146e9b681f171/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.375343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377789bc-12a3-496d-a29b-16d62f5fd64c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ba5a78562b439ebb0667c0f09c2507, user_id=a0b353f4aae24694afb146e9b681f171, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:54.375343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377789bc-12a3-496d-a29b-16d62f5fd64c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:54.375907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fe32f-11b3-4790-8074-5810327e10e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:54.377717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 822/1633] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:04:54 2022] PUT /identity/v3/projects/a4ba5a78562b439ebb0667c0f09c2507/users/a0b353f4aae24694afb146e9b681f171/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:04:54.380291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fe32f-11b3-4790-8074-5810327e10e1 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:54.380904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e81d05-2416-4190-9058-da3e0e3ab2f5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.380904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e81d05-2416-4190-9058-da3e0e3ab2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.380904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e81d05-2416-4190-9058-da3e0e3ab2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.384328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 812/1634] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:54.384328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8f6c0-2fc3-49ef-bd7e-4834d0659a9f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.384328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8f6c0-2fc3-49ef-bd7e-4834d0659a9f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.384328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8f6c0-2fc3-49ef-bd7e-4834d0659a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.390341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b3e81d05-2416-4190-9058-da3e0e3ab2f5 None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:54.390341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b3e81d05-2416-4190-9058-da3e0e3ab2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:54.391653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-55f8f6c0-2fc3-49ef-bd7e-4834d0659a9f None None] MFA Rules not processed for user `a0b353f4aae24694afb146e9b681f171`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:54.391653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-55f8f6c0-2fc3-49ef-bd7e-4834d0659a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:54.392160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 823/1635] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:04:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:54.392160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e81d05-2416-4190-9058-da3e0e3ab2f5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:54.402518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 813/1636] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:54.455815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:54.464545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e81d05-2416-4190-9058-da3e0e3ab2f5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:54.468669 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8f6c0-2fc3-49ef-bd7e-4834d0659a9f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:54.480823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8f6c0-2fc3-49ef-bd7e-4834d0659a9f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:54.484300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd1627-3c18-434e-97ef-1181c4c31456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:54.485406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd1627-3c18-434e-97ef-1181c4c31456 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.486274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd1627-3c18-434e-97ef-1181c4c31456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.487013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd1627-3c18-434e-97ef-1181c4c31456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.489072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:54.490394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:54.491772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:54.492164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:54.512732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd1627-3c18-434e-97ef-1181c4c31456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:54.515341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd1627-3c18-434e-97ef-1181c4c31456 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:54.515594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:54.517255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:54.553209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 814/1637] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:04:54.571299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 824/1638] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:04:54.707867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:55.122990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:55.134966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca7545-2dfa-4eb3-a5d7-5b51cd3ecb62 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:55.142741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dbba3-a71e-4461-a8a3-8c1c677baa06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:55.143362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dbba3-a71e-4461-a8a3-8c1c677baa06 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:55.143689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dbba3-a71e-4461-a8a3-8c1c677baa06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:55.144236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dbba3-a71e-4461-a8a3-8c1c677baa06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:55.251948 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dbba3-a71e-4461-a8a3-8c1c677baa06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:55.253890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dbba3-a71e-4461-a8a3-8c1c677baa06 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:55.283254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 815/1639] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:56.525212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497813-b8c4-4a6f-baf7-01da9436996c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:56.525776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497813-b8c4-4a6f-baf7-01da9436996c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:56.525926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497813-b8c4-4a6f-baf7-01da9436996c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:56.527949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 825/1640] 10.222.0.72 () {54 vars in 962 bytes} [Tue Sep 13 13:04:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:04:56.540703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3136218-9abe-4bf7-9ea2-557cdcd1287f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:56.541138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3136218-9abe-4bf7-9ea2-557cdcd1287f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:56.541910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3136218-9abe-4bf7-9ea2-557cdcd1287f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:56.580490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d3136218-9abe-4bf7-9ea2-557cdcd1287f None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:56.582197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d3136218-9abe-4bf7-9ea2-557cdcd1287f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:56.594416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 816/1641] 10.222.0.72 () {58 vars in 1059 bytes} [Tue Sep 13 13:04:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:56.600189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357869-8c28-44d4-af55-b1afa369f2d6 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:56.602201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357869-8c28-44d4-af55-b1afa369f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:56.605657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357869-8c28-44d4-af55-b1afa369f2d6 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:56.605657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 826/1642] 10.222.0.72 () {54 vars in 912 bytes} [Tue Sep 13 13:04:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:04:56.614780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489cc5b-7fe3-4695-a873-98519571cfb0 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:56.617518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489cc5b-7fe3-4695-a873-98519571cfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:56.617831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489cc5b-7fe3-4695-a873-98519571cfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:56.678873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5489cc5b-7fe3-4695-a873-98519571cfb0 None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:56.680351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5489cc5b-7fe3-4695-a873-98519571cfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:56.701899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 817/1643] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:04:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:56.716218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357869-8c28-44d4-af55-b1afa369f2d6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:56.733474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357869-8c28-44d4-af55-b1afa369f2d6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:56.747389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc078d-110d-4d04-b0b0-89909b97a6a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:56.748128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc078d-110d-4d04-b0b0-89909b97a6a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:56.748476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc078d-110d-4d04-b0b0-89909b97a6a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:56.749423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc078d-110d-4d04-b0b0-89909b97a6a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:56.765718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc078d-110d-4d04-b0b0-89909b97a6a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:56.766845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc078d-110d-4d04-b0b0-89909b97a6a5 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:56.780198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 827/1644] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:58.244603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ca8ed-6e08-4b87-8b32-a30dc0ec395b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:58.245146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ca8ed-6e08-4b87-8b32-a30dc0ec395b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:58.245146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ca8ed-6e08-4b87-8b32-a30dc0ec395b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:58.258322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:04:58.338662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2a6ca8ed-6e08-4b87-8b32-a30dc0ec395b None None] MFA Rules not processed for user `e54f17d6995f4c099708d79715d95220`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:58.340243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2a6ca8ed-6e08-4b87-8b32-a30dc0ec395b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:58.483054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 818/1645] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:58.497954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564bd49-b7e3-43f0-a0d1-e57df35fb6e4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:58.499681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564bd49-b7e3-43f0-a0d1-e57df35fb6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:58.500141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564bd49-b7e3-43f0-a0d1-e57df35fb6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:58.571121 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3564bd49-b7e3-43f0-a0d1-e57df35fb6e4 None None] MFA Rules not processed for user `e54f17d6995f4c099708d79715d95220`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:58.575814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3564bd49-b7e3-43f0-a0d1-e57df35fb6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:58.603448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 828/1646] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:04:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:04:58.638117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ca8ed-6e08-4b87-8b32-a30dc0ec395b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:04:58.666283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ca8ed-6e08-4b87-8b32-a30dc0ec395b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:04:58.674459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d115e4-a6ed-4695-935d-6a55a28a2ec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:04:58.675328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d115e4-a6ed-4695-935d-6a55a28a2ec6 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:58.676152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d115e4-a6ed-4695-935d-6a55a28a2ec6 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:58.676585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d115e4-a6ed-4695-935d-6a55a28a2ec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:58.688152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d115e4-a6ed-4695-935d-6a55a28a2ec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:04:58.690934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d115e4-a6ed-4695-935d-6a55a28a2ec6 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:04:58.716387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 819/1647] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:04:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:04:59.780382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393fad-2192-493e-8d12-f1488b3edfb1 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:59.781262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393fad-2192-493e-8d12-f1488b3edfb1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:59.781682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393fad-2192-493e-8d12-f1488b3edfb1 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:59.783693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 829/1648] 10.222.0.72 () {54 vars in 912 bytes} [Tue Sep 13 13:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:04:59.790351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d280-1aa9-4fda-acd5-4ef4ac8259c1 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:04:59.790893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d280-1aa9-4fda-acd5-4ef4ac8259c1 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:04:59.791093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d280-1aa9-4fda-acd5-4ef4ac8259c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:04:59.880276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5a58d280-1aa9-4fda-acd5-4ef4ac8259c1 None None] MFA Rules not processed for user `4ed50a15319e4991ae90b0e5271f4c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:04:59.881034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5a58d280-1aa9-4fda-acd5-4ef4ac8259c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:04:59.913639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26393fad-2192-493e-8d12-f1488b3edfb1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:00.057391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 820/1649] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:04:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:00.059527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d280-1aa9-4fda-acd5-4ef4ac8259c1 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:00.076298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d280-1aa9-4fda-acd5-4ef4ac8259c1 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:00.086422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:00.087268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:00.087552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:00.087830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:00.114529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:00.118309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:00.130488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26393fad-2192-493e-8d12-f1488b3edfb1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:00.135875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 821/1650] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:05:00.144233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:00.144775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:00.145300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:00.145954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] PATH_INFO: `/v3/users/f76a56d0dbeb43d1b0e2d87b1a27544c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:00.148297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76a56d0dbeb43d1b0e2d87b1a27544c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:00.150419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:00.495047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76a56d0dbeb43d1b0e2d87b1a27544c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:00.496238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76a56d0dbeb43d1b0e2d87b1a27544c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:00.507882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76a56d0dbeb43d1b0e2d87b1a27544c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:00.530589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76a56d0dbeb43d1b0e2d87b1a27544c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:00.531794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b405a42d-04a9-42b7-a44e-8faaf48ebc7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76a56d0dbeb43d1b0e2d87b1a27544c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:00.635866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:00.653081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4a18-105b-48ac-a364-21d0c3e196be service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:00.664120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:00.667629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:00.667629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:00.667629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] PATH_INFO: `/v3/users/a2eb560030db496bb256054e6c50a32e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:00.670426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2eb560030db496bb256054e6c50a32e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:00.671740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: WARNING py.warnings [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:05:00.671740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: warnings.warn(msg) Sep 13 13:05:00.671740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: Sep 13 13:05:00.672954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:00.712556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 830/1651] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:04:59 2022] DELETE /identity/v3/users/f76a56d0dbeb43d1b0e2d87b1a27544c => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:00.717073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:00.717490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:00.718271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:00.748685 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:00.760449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:00.763842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:00.916383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 831/1652] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:00.977950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2eb560030db496bb256054e6c50a32e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:00.988957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2eb560030db496bb256054e6c50a32e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:01.012341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2eb560030db496bb256054e6c50a32e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:01.014604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:01.044309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:01.048716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:01.057053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2eb560030db496bb256054e6c50a32e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:01.057839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98636b31-cb8c-4d61-9d3f-ef06663dde7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2eb560030db496bb256054e6c50a32e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:01.183117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad013c2-a6d1-49f4-9b93-030e4f80e20a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:01.196657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:01.198479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:01.202247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:01.202247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:01.202247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:01.261321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:01.264306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 822/1653] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:00 2022] DELETE /identity/v3/users/a2eb560030db496bb256054e6c50a32e => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:01.273084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:01.273310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:01.273531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:01.274037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:01.339591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 832/1654] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:01.354461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:01.364851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:01.530269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:01.531804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:01.599371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 823/1655] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:01.650227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:01.651994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:01.780861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd07928-c8b8-45c7-bdf4-a40faa5aef79 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:01.867284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f452d38-cb78-4a9f-81e1-b7aa95eef5e8 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:01.882265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:01.885278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:01.885613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:01.887624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:01.925472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:01.927335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:01.959517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 824/1656] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:01.969405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:02.002478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:02.003116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:02.003479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:02.003759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:02.007375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:02.067924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:02.076287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:02.121701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 833/1657] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:03.438937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:03.438937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:03.461099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dc864-56bf-4265-9786-b1153899a261 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:03.471122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:03.471780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:03.472241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:03.472670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] PATH_INFO: `/v3/projects/f38ac749002a4725a20b4d0037f33952` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:03.476576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f38ac749002a4725a20b4d0037f33952)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:03.478803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:03.669647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f38ac749002a4725a20b4d0037f33952', 'request_id': 'req-0f0aaf1e-d15f-4b55-94d6-c2718577d295'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:03.764616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f38ac749002a4725a20b4d0037f33952', 'request_id': 'req-0f0aaf1e-d15f-4b55-94d6-c2718577d295'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:03.776452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f38ac749002a4725a20b4d0037f33952', 'request_id': 'req-0f0aaf1e-d15f-4b55-94d6-c2718577d295'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:03.790548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:03.793426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] The token cache is being invalidate because project f38ac749002a4725a20b4d0037f33952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:03.793979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:03.794902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:03.797684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 825/1658] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:03 2022] DELETE /identity/v3/projects/f38ac749002a4725a20b4d0037f33952 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:03.801837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:03.816452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abb2ec-a5f3-4b91-8a89-2e92b7847d47 service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:03.910097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:03.910844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:03.911133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:03.911429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] PATH_INFO: `/v3/users/2d3af52c82614b5396a14c760c8be6a8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:03.913365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3af52c82614b5396a14c760c8be6a8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:03.915957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:04.182288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3af52c82614b5396a14c760c8be6a8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:04.183014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3af52c82614b5396a14c760c8be6a8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:04.216126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3af52c82614b5396a14c760c8be6a8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:04.256086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3af52c82614b5396a14c760c8be6a8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:04.257392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6151b05a-0af0-416c-b73c-87c8d4e67e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3af52c82614b5396a14c760c8be6a8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:04.360635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 834/1659] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:03 2022] DELETE /identity/v3/users/2d3af52c82614b5396a14c760c8be6a8 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:04.717046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:04.718620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:04.763570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aaf1e-d15f-4b55-94d6-c2718577d295 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:04.933864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:04.933864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:04.933864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:04.933864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] PATH_INFO: `/v3/projects/439cc533cdb04e508aeb56243998d880` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:04.934305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=439cc533cdb04e508aeb56243998d880)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:04.938573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:04.983832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:04.985067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:04.985651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:05.023709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] MFA Rules not processed for user `9fadb80f21614816b3526d59b2d8a0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:05.025324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:05.130237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:05.178633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 835/1660] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:05.179203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '439cc533cdb04e508aeb56243998d880', 'request_id': 'req-9d844f0b-d935-43dd-983c-b3aeb379b981'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.186562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:05.188310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:05.314931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28d5f5-ee99-4712-8678-e8ee507178f1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:05.335842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '439cc533cdb04e508aeb56243998d880', 'request_id': 'req-9d844f0b-d935-43dd-983c-b3aeb379b981'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.345179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '439cc533cdb04e508aeb56243998d880', 'request_id': 'req-9d844f0b-d935-43dd-983c-b3aeb379b981'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.370683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] The token cache is being invalidate because project 439cc533cdb04e508aeb56243998d880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:05.371605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.372126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9d844f0b-d935-43dd-983c-b3aeb379b981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.374966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 826/1661] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:04 2022] DELETE /identity/v3/projects/439cc533cdb04e508aeb56243998d880 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:05.381408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:05.382709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:05.383384 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:05.439370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:05.440786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:05.488446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b86fdf-e14d-4004-b5f3-f2f9a4bc0034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:05.489146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b86fdf-e14d-4004-b5f3-f2f9a4bc0034 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:05.489558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b86fdf-e14d-4004-b5f3-f2f9a4bc0034 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:05.490000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b86fdf-e14d-4004-b5f3-f2f9a4bc0034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:05.495151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c5b86fdf-e14d-4004-b5f3-f2f9a4bc0034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:05.593980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 827/1662] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:05.612457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:05.612457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:05.626499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead32fa-2055-4bd6-9c24-a15cf482ca75 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:05.647295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:05.648523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:05.648854 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:05.649070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] PATH_INFO: `/v3/projects/b1635a8e42be4165a09715645943df03` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:05.650582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1635a8e42be4165a09715645943df03)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:05.655694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:05.678095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b86fdf-e14d-4004-b5f3-f2f9a4bc0034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:05.680986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b86fdf-e14d-4004-b5f3-f2f9a4bc0034 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:05.716258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 836/1663] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:05:05.720225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997808-a8a9-43cd-a61c-2745342b6389 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:05.720515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997808-a8a9-43cd-a61c-2745342b6389 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:05.720794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997808-a8a9-43cd-a61c-2745342b6389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:05.772116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-10997808-a8a9-43cd-a61c-2745342b6389 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:05.773612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-10997808-a8a9-43cd-a61c-2745342b6389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:05.799830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 837/1664] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:05.805709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10997808-a8a9-43cd-a61c-2745342b6389 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:05.838770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10997808-a8a9-43cd-a61c-2745342b6389 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:05.841652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:05.860009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:05.862067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:05.862491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:05.862773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:05.863446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:05.863751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:05.872498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1635a8e42be4165a09715645943df03', 'request_id': 'req-7e341f3f-24a1-4776-9963-84cf4bd828a3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.876569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:05.904963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 838/1665] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:05:05.905958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:05.922577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a325a5-4322-4b80-ab6d-67f0257b81ea admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:05.947304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1635a8e42be4165a09715645943df03', 'request_id': 'req-7e341f3f-24a1-4776-9963-84cf4bd828a3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.962680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1635a8e42be4165a09715645943df03', 'request_id': 'req-7e341f3f-24a1-4776-9963-84cf4bd828a3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.993329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] The token cache is being invalidate because project b1635a8e42be4165a09715645943df03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:05.994388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:05.997868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7e341f3f-24a1-4776-9963-84cf4bd828a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:06.003130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 828/1666] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:05 2022] DELETE /identity/v3/projects/b1635a8e42be4165a09715645943df03 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:06.012124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5029-98ca-4665-befd-1822edff2350 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.012124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5029-98ca-4665-befd-1822edff2350 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.012124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f5029-98ca-4665-befd-1822edff2350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.058799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-883f5029-98ca-4665-befd-1822edff2350 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:06.060179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-883f5029-98ca-4665-befd-1822edff2350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:06.094300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.096242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.097090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.097379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.099211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.106049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:06.163284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 829/1667] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:06.172532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16a15e-fa08-4f63-906a-6efa01ce55e5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.173030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16a15e-fa08-4f63-906a-6efa01ce55e5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.173599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16a15e-fa08-4f63-906a-6efa01ce55e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.207253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 839/1668] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:05 2022] POST /identity/v3/projects => generated 371 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:06.212314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.214394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ce16a15e-fa08-4f63-906a-6efa01ce55e5 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:06.215595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:06.217341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ce16a15e-fa08-4f63-906a-6efa01ce55e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:06.322414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 830/1669] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:06.323448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16a15e-fa08-4f63-906a-6efa01ce55e5 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.336969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16a15e-fa08-4f63-906a-6efa01ce55e5 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.343221 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.344295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.344801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.345278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.346198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:06.346894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.348869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:06.370233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 831/1670] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:05:06.376122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.390307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6ccc2-ccb4-4680-beae-32602116300f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.409904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55301bf-99c6-4b4b-b09f-52cd640fc341 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.415914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.416710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.417079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.417450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.418301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.423515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:06.444942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.445574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.445876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.446148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.446888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.449882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:06.544613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 832/1671] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:06 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:06.555310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.564057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 840/1672] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:06 2022] POST /identity/v3/users => generated 286 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:06.565457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.588663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22cfba-f98d-482a-b950-8e81ac797323 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.702007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:06.759661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac35e-13be-4711-9f92-5c73aea93f88 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.767117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.768082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.768998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.769469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.770462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.775184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:06.797378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.798638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.798925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.799202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.800534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.806196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:06.824371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 841/1673] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:06.826133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.875148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e411d82-d9fd-471e-b6c8-6cf277fa4b1a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.890614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.892694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.892889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.893176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.894169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.900148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:06.930734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 842/1674] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:06.935235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.941187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 833/1675] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:06 2022] POST /identity/v3/users => generated 289 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:06.946218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:06.953888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817b50f-e7fa-4ba1-a64e-3f6a6a65cf1f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.960977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e464f45-533e-4430-9440-fe0cd21f27bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.962925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464f45-533e-4430-9440-fe0cd21f27bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.963671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464f45-533e-4430-9440-fe0cd21f27bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.964584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464f45-533e-4430-9440-fe0cd21f27bc admin admin] PATH_INFO: `/v3/projects/d3dd03463b6045f4b6f25f5da09df2d1/users/bde5f1c0b00248d79c77fde6f41f0b43/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.980306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac0cd-87ed-44f3-8bb9-763d750ccba6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:06.991402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:06.993236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:06.993236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:06.993236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:06.993524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:06.996022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:07.007630 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e464f45-533e-4430-9440-fe0cd21f27bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3dd03463b6045f4b6f25f5da09df2d1, user_id=bde5f1c0b00248d79c77fde6f41f0b43, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:07.012670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e464f45-533e-4430-9440-fe0cd21f27bc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:07.019342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 834/1676] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:05:07.032231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:07.046542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a2a0d-f2da-47d8-a38e-a81aa0a70aa2 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:07.062254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:07.062896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:07.063215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:07.065008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:07.065602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:07.067277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:07.101229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 835/1677] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:07.101448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:07.118437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 843/1678] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:06 2022] PUT /identity/v3/projects/d3dd03463b6045f4b6f25f5da09df2d1/users/bde5f1c0b00248d79c77fde6f41f0b43/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:07.121006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51f70-3fc9-4ac5-ad6c-ca5382fe656c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:07.121311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51f70-3fc9-4ac5-ad6c-ca5382fe656c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:07.121574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f51f70-3fc9-4ac5-ad6c-ca5382fe656c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:07.130616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b732-6710-43b0-af30-f1c0872b2ccc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:07.170180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:07.189162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-42f51f70-3fc9-4ac5-ad6c-ca5382fe656c None None] MFA Rules not processed for user `bde5f1c0b00248d79c77fde6f41f0b43`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:07.194717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-42f51f70-3fc9-4ac5-ad6c-ca5382fe656c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:07.271117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b3a2f-55c1-4d66-851d-f1cbe934e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:07.271811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b3a2f-55c1-4d66-851d-f1cbe934e0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:07.272131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b3a2f-55c1-4d66-851d-f1cbe934e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:07.272411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b3a2f-55c1-4d66-851d-f1cbe934e0d7 admin admin] PATH_INFO: `/v3/projects/4d0b8e1853944a739c91e03c0d235682/users/715eab47227449cbaa5c00d0fb609647/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:07.300028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b3a2f-55c1-4d66-851d-f1cbe934e0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0b8e1853944a739c91e03c0d235682, user_id=715eab47227449cbaa5c00d0fb609647, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:07.302963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b3a2f-55c1-4d66-851d-f1cbe934e0d7 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:07.376743 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 844/1679] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:07.378413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f51f70-3fc9-4ac5-ad6c-ca5382fe656c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:07.402668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 836/1680] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:07 2022] PUT /identity/v3/projects/4d0b8e1853944a739c91e03c0d235682/users/715eab47227449cbaa5c00d0fb609647/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:07.406557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:07.406955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f51f70-3fc9-4ac5-ad6c-ca5382fe656c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:07.418840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b853cf-7f60-4e85-ade1-3e34edf038ae None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:07.419130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b853cf-7f60-4e85-ade1-3e34edf038ae None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:07.419409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b853cf-7f60-4e85-ade1-3e34edf038ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:07.493862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-78b853cf-7f60-4e85-ade1-3e34edf038ae None None] MFA Rules not processed for user `715eab47227449cbaa5c00d0fb609647`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:07.495235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-78b853cf-7f60-4e85-ade1-3e34edf038ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:07.627406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:07.628326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:07.628910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:07.629329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:07.634955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:07.637377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:07.765666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 837/1681] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:07.768310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b68c58-9584-41b8-b0b4-3304f582e113 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:07.768605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b68c58-9584-41b8-b0b4-3304f582e113 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:07.768881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b68c58-9584-41b8-b0b4-3304f582e113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:07.846962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 845/1682] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:07 2022] POST /identity/v3/projects => generated 371 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:07.848042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:07.907224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-73b68c58-9584-41b8-b0b4-3304f582e113 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:07.910164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-73b68c58-9584-41b8-b0b4-3304f582e113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:08.074330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:08.078650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae11d2-7e49-46db-ae74-076d7e8b2142 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.094838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.095405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.101578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.101578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.101578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.101578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.209308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 838/1683] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:05:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:08.211265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b68c58-9584-41b8-b0b4-3304f582e113 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:08.249471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b68c58-9584-41b8-b0b4-3304f582e113 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.261927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.263396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.266425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.266425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.266425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.270305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.340999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 846/1684] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:07 2022] POST /identity/v3/projects => generated 375 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:08.342037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:08.469013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 839/1685] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:08 2022] POST /identity/v3/users => generated 292 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:08.474728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:08.506983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fa05a-0991-4860-aa44-0f5313edecc2 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.524071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4fb23-a9da-4395-b617-5aa955873bc4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.529617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.533756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.535185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.535185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.536855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.545137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.557208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.557208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.557417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.557593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.558280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.560642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.585917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 840/1686] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:08.587086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:08.628717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196175d-24a8-4e4d-84b8-1d7adbd71035 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.648634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.650319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.661988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.661988 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] PATH_INFO: `/v3/projects/80b10694cd934436be4f0e85b265a701/users/b68a02f8890c4b0384fda1f431452664/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.684547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b10694cd934436be4f0e85b265a701, user_id=b68a02f8890c4b0384fda1f431452664, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.695292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.707817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 847/1687] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:08 2022] POST /identity/v3/users => generated 294 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:08.712058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:08.745825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50433dad-0c25-49aa-bdc0-60dd080449dd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.760887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:08.766276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.766903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.767190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.767468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.768131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.769762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.816349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 848/1688] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:05:08.821019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:08.833063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cd8e2-6211-4348-88d5-2609922c6f85 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.844966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.847674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.848067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.848412 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] PATH_INFO: `/v3/projects/7e2e2b414acd4accba56b193c2fc3952/users/0c8eb43f61404fc28d94c319d3e33d13/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.868359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 841/1689] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:08 2022] PUT /identity/v3/projects/80b10694cd934436be4f0e85b265a701/users/b68a02f8890c4b0384fda1f431452664/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:08.869818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:08.883214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2e2b414acd4accba56b193c2fc3952, user_id=0c8eb43f61404fc28d94c319d3e33d13, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.883451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.967050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f919a-d765-468a-b1ae-355165eec6ad admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:08.972658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:08.974139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:08.974554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:08.974936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:08.975990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:08.978878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:08.994399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 842/1690] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:08.999948 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:09.002571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:09.030760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 849/1691] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:08 2022] PUT /identity/v3/projects/7e2e2b414acd4accba56b193c2fc3952/users/0c8eb43f61404fc28d94c319d3e33d13/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:09.030760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:09.227247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3fa6c-e5ca-4fc6-a67b-2cbee3d77038 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:09.252118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49eed1-ee69-4351-93e1-104140c1dd76 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:09.266473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:09.267428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:09.267832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:09.268195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] PATH_INFO: `/v3/projects/80b10694cd934436be4f0e85b265a701/users/b68a02f8890c4b0384fda1f431452664/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:09.271402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b10694cd934436be4f0e85b265a701, user_id=b68a02f8890c4b0384fda1f431452664, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:09.278269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:09.324822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 850/1692] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:09 2022] PUT /identity/v3/projects/80b10694cd934436be4f0e85b265a701/users/b68a02f8890c4b0384fda1f431452664/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:09.326607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:09.458757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:09.487111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:09.487813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:09.490120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:09.490484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:09.529566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:09.550491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb316e4-8b46-4e0d-b44c-51c277babd55 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:09.563328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7df32-1f7e-4020-9aff-adb1a41b152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:09.565052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7df32-1f7e-4020-9aff-adb1a41b152f admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:09.565338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7df32-1f7e-4020-9aff-adb1a41b152f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:09.565651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7df32-1f7e-4020-9aff-adb1a41b152f admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:09.566331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7df32-1f7e-4020-9aff-adb1a41b152f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:09.575063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7df32-1f7e-4020-9aff-adb1a41b152f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:09.617025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 851/1693] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:09.624513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:09.626464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:09.628614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:09.629030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:09.629323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:09.656374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 843/1694] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:05:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:05:09.657560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:09.706655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36859ead-1374-4dce-bd9a-4cd0e309ec44 service swift] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:09.724625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859c3b5-c334-463a-a4a1-a247ef346fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:09.729718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859c3b5-c334-463a-a4a1-a247ef346fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:09.730235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859c3b5-c334-463a-a4a1-a247ef346fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:09.734916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859c3b5-c334-463a-a4a1-a247ef346fcb admin admin] PATH_INFO: `/v3/projects/7e2e2b414acd4accba56b193c2fc3952/users/0c8eb43f61404fc28d94c319d3e33d13/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:09.735953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859c3b5-c334-463a-a4a1-a247ef346fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2e2b414acd4accba56b193c2fc3952, user_id=0c8eb43f61404fc28d94c319d3e33d13, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:09.742139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859c3b5-c334-463a-a4a1-a247ef346fcb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:09.750365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] MFA Rules not processed for user `b68a02f8890c4b0384fda1f431452664`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:09.751655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:09.776507 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 844/1695] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:09 2022] PUT /identity/v3/projects/7e2e2b414acd4accba56b193c2fc3952/users/0c8eb43f61404fc28d94c319d3e33d13/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:09.792569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:09.793081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:09.793523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:09.878428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] MFA Rules not processed for user `0c8eb43f61404fc28d94c319d3e33d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:09.879729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:10.015338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 852/1696] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:10.029699 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:10.031549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:10.130938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 845/1697] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:10.142393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:10.143947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:10.253540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabde7de-0ebb-4821-8070-74a6e5b027f7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:10.374768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1063c2-a679-4489-b4a2-e58bfc4a0cfb None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:10.391799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:10.392795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:10.395813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:10.398108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:10.405308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:10.486485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cde66a-d1f0-4cb8-943c-a9bfb9c66186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:10.488468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cde66a-d1f0-4cb8-943c-a9bfb9c66186 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:10.490174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cde66a-d1f0-4cb8-943c-a9bfb9c66186 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:10.492784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cde66a-d1f0-4cb8-943c-a9bfb9c66186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:10.568581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:10.570799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:10.589107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:10.603373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 846/1698] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:10.817823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cde66a-d1f0-4cb8-943c-a9bfb9c66186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:10.819785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cde66a-d1f0-4cb8-943c-a9bfb9c66186 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:10.875824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 853/1699] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 842 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:05:12.309378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:12.321588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f381268-3f2e-42a6-a2fe-05398f4552ee service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:12.332527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:12.333304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:12.334155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:12.335584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:12.677773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:12.680188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:12.710153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 847/1700] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:13.714347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496679-4138-4e91-9876-2c666e1d5138 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:13.714347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496679-4138-4e91-9876-2c666e1d5138 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:13.714347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496679-4138-4e91-9876-2c666e1d5138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:13.801231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5a496679-4138-4e91-9876-2c666e1d5138 None None] MFA Rules not processed for user `fe3be97820d64f85aa4d2dbd8f3ffd33`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:13.803307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a496679-4138-4e91-9876-2c666e1d5138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:14.008780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 854/1701] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:14.021643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:14.023437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:14.035337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:14.247789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a837e-c3dc-412f-8008-a20eafda125f service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:14.268219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:14.268219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:14.268219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:14.268219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:14.270041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210bf52-d2a7-416c-9ca6-1411291c838f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:14.270417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210bf52-d2a7-416c-9ca6-1411291c838f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:14.270805 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210bf52-d2a7-416c-9ca6-1411291c838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:14.297579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:14.299342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:14.312486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5210bf52-d2a7-416c-9ca6-1411291c838f None None] MFA Rules not processed for user `4ed50a15319e4991ae90b0e5271f4c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:14.319126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5210bf52-d2a7-416c-9ca6-1411291c838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:14.329359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 848/1702] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:14.505692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 855/1703] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Sep 13 13:05:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:14.574269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:14.576084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:14.611011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529091e4-212b-4dd8-b55f-c47c6d325f3d service nova] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:14.626476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac168bf-dc92-41a4-a7ad-f9282f25b65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:14.626476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac168bf-dc92-41a4-a7ad-f9282f25b65a service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:14.626476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac168bf-dc92-41a4-a7ad-f9282f25b65a service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:14.627281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac168bf-dc92-41a4-a7ad-f9282f25b65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:14.693476 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac168bf-dc92-41a4-a7ad-f9282f25b65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:14.696864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac168bf-dc92-41a4-a7ad-f9282f25b65a service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:14.726441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 849/1704] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:14.764264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c62ba-6a0b-47bf-8333-33b715182ef2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:14.766062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c62ba-6a0b-47bf-8333-33b715182ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:14.766426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c62ba-6a0b-47bf-8333-33b715182ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:14.841566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5c4c62ba-6a0b-47bf-8333-33b715182ef2 None None] MFA Rules not processed for user `9fadb80f21614816b3526d59b2d8a0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:14.845643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5c4c62ba-6a0b-47bf-8333-33b715182ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:15.088794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 856/1705] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:19.095769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2db4-ac32-4f5a-b100-5bdeb2f49181 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:19.096240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2db4-ac32-4f5a-b100-5bdeb2f49181 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:19.096312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b2db4-ac32-4f5a-b100-5bdeb2f49181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:19.148971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-085b2db4-ac32-4f5a-b100-5bdeb2f49181 None None] MFA Rules not processed for user `fe3be97820d64f85aa4d2dbd8f3ffd33`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:19.150276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-085b2db4-ac32-4f5a-b100-5bdeb2f49181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:19.173239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 850/1706] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:19.531541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd9524-5e98-4a15-8471-303e8718cb38 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:19.532096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd9524-5e98-4a15-8471-303e8718cb38 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:19.532172 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd9524-5e98-4a15-8471-303e8718cb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:19.543257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:19.618104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8cfd9524-5e98-4a15-8471-303e8718cb38 None None] MFA Rules not processed for user `7b1c15ff53e5442881b56bf504b9a31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:19.618104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8cfd9524-5e98-4a15-8471-303e8718cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:19.894164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 857/1707] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:20.021002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2db4-ac32-4f5a-b100-5bdeb2f49181 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:20.034479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b2db4-ac32-4f5a-b100-5bdeb2f49181 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:20.066410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6ca5a-1d9c-452d-beed-3e57c6527046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:20.067427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6ca5a-1d9c-452d-beed-3e57c6527046 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:20.067947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6ca5a-1d9c-452d-beed-3e57c6527046 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:20.068425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6ca5a-1d9c-452d-beed-3e57c6527046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:20.097575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6ca5a-1d9c-452d-beed-3e57c6527046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:20.099888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6ca5a-1d9c-452d-beed-3e57c6527046 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:20.127878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 851/1708] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:21.270107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87a0ab-5c91-4587-8b92-9aef40228774 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:21.270652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87a0ab-5c91-4587-8b92-9aef40228774 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:21.271022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87a0ab-5c91-4587-8b92-9aef40228774 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:21.272859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 858/1709] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:05:21.519691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:21.520641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:21.521175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:21.561648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb2273-8a7d-4fae-8fc9-826ed08d7f6c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:21.562400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb2273-8a7d-4fae-8fc9-826ed08d7f6c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:21.562787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb2273-8a7d-4fae-8fc9-826ed08d7f6c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:21.565065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 859/1710] 10.222.0.72 () {54 vars in 912 bytes} [Tue Sep 13 13:05:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:05:21.569344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:21.572844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:21.574337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d2952-f95e-4177-b3d0-711d96f35722 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:21.574734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d2952-f95e-4177-b3d0-711d96f35722 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:21.575116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d2952-f95e-4177-b3d0-711d96f35722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:21.603974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 852/1711] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:05:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:21.642271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:21.644543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:21.660374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7e0d2952-f95e-4177-b3d0-711d96f35722 None None] MFA Rules not processed for user `4ed50a15319e4991ae90b0e5271f4c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:21.664778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7e0d2952-f95e-4177-b3d0-711d96f35722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:21.683871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 860/1712] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:05:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:21.768557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d2952-f95e-4177-b3d0-711d96f35722 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:21.771759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ed4c-ec6e-4075-96be-4c5c4ddea63c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:21.777145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:21.777823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:21.778431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:21.778712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] PATH_INFO: `/v3/users/0c80935a0ae9472ab49c1e72cac50f25` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:21.782526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c80935a0ae9472ab49c1e72cac50f25)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:21.782679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d2952-f95e-4177-b3d0-711d96f35722 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:21.785187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:21.791979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:21.792890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:21.793310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:21.793715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:21.803945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:21.806081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:21.819995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 861/1713] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:21.994214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c80935a0ae9472ab49c1e72cac50f25'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:22.009634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:22.012338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c80935a0ae9472ab49c1e72cac50f25'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:22.018570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:22.023072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c80935a0ae9472ab49c1e72cac50f25'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:22.033446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1d163-8156-4218-afd7-9b14ee2e180f service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:22.042635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ac3f-d627-4d8b-b479-0401e0a2d117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:22.043657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ac3f-d627-4d8b-b479-0401e0a2d117 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:22.044467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ac3f-d627-4d8b-b479-0401e0a2d117 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:22.044881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ac3f-d627-4d8b-b479-0401e0a2d117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:22.045222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c80935a0ae9472ab49c1e72cac50f25'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:22.046563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89daebfe-5d39-4a9b-9d61-f37c4d79f04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c80935a0ae9472ab49c1e72cac50f25'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:22.047416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c4f0ac3f-d627-4d8b-b479-0401e0a2d117 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.062094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0ac3f-d627-4d8b-b479-0401e0a2d117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:22.064253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0ac3f-d627-4d8b-b479-0401e0a2d117 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:22.076602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 862/1714] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:22.078116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf54ad-e797-46dc-8d4a-a49d16b8473b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:22.078537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf54ad-e797-46dc-8d4a-a49d16b8473b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:22.078944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf54ad-e797-46dc-8d4a-a49d16b8473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:22.126547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 853/1715] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:21 2022] DELETE /identity/v3/users/0c80935a0ae9472ab49c1e72cac50f25 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:22.130843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:22.131947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:22.132618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:22.134768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-26cf54ad-e797-46dc-8d4a-a49d16b8473b None None] MFA Rules not processed for user `8c688fff323943b3bbcbb0569d974d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:22.136013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-26cf54ad-e797-46dc-8d4a-a49d16b8473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.175058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:22.176434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.260093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 863/1716] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:22.264127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186015ab-7355-4909-b392-d5c724842b3a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:22.264419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186015ab-7355-4909-b392-d5c724842b3a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:22.269067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186015ab-7355-4909-b392-d5c724842b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:22.272523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 854/1717] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:22.282964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:22.284477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.346849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-186015ab-7355-4909-b392-d5c724842b3a None None] MFA Rules not processed for user `8c688fff323943b3bbcbb0569d974d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:22.354389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-186015ab-7355-4909-b392-d5c724842b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.377512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 864/1718] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:22.380799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c837c-ae77-4bd1-813f-c047780c6e45 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:22.396256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:22.398872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:22.398872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:22.398872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:22.410258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:22.412768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:22.446068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 855/1719] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:22.509093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186015ab-7355-4909-b392-d5c724842b3a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:22.510913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-186015ab-7355-4909-b392-d5c724842b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.574951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:22.655701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186015ab-7355-4909-b392-d5c724842b3a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:22.667134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:22.667775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:22.668575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:22.669409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:22.672093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.676620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:22.678653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:22.687178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:22.688847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:22.697790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70482ecc-9005-4269-a38b-c5296767d144 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:22.702885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 865/1720] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:22.710006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:22.711361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:22.711997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:22.712542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:22.723812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:22.726119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:22.737985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 856/1721] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:05:23.581011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:23.582721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:23.606193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee63bd3-3012-4b42-bc42-d73cd7df449b service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:23.613717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:23.615192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:23.615894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:23.616503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] PATH_INFO: `/v3/projects/59ba2777b165453ebbafa35810212015` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:23.618260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ba2777b165453ebbafa35810212015)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:23.619492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: WARNING py.warnings [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 13:05:23.619492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: warnings.warn(msg) Sep 13 13:05:23.619492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: Sep 13 13:05:23.621706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:23.828678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ba2777b165453ebbafa35810212015', 'request_id': 'req-5eaeead1-4050-47a1-a98d-ac9e64a287ba'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:23.895575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ba2777b165453ebbafa35810212015', 'request_id': 'req-5eaeead1-4050-47a1-a98d-ac9e64a287ba'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:23.920068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ba2777b165453ebbafa35810212015', 'request_id': 'req-5eaeead1-4050-47a1-a98d-ac9e64a287ba'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:23.935770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] The token cache is being invalidate because project 59ba2777b165453ebbafa35810212015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:23.936360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:23.937054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5eaeead1-4050-47a1-a98d-ac9e64a287ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:23.938690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 866/1722] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:23 2022] DELETE /identity/v3/projects/59ba2777b165453ebbafa35810212015 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:23.944107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:23.966159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768d43b-efa2-4655-af67-3b834841234a service nova] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:24.236850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:24.237875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:24.238326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:24.239617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] PATH_INFO: `/v3/users/a0b353f4aae24694afb146e9b681f171` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:24.243398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b353f4aae24694afb146e9b681f171)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:24.246564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:24.249167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:24.390213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:24.390638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:24.390784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:24.434018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] MFA Rules not processed for user `8c688fff323943b3bbcbb0569d974d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:24.435179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:24.562415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b353f4aae24694afb146e9b681f171'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:24.566176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b353f4aae24694afb146e9b681f171'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:24.579426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 867/1723] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:24.585964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b353f4aae24694afb146e9b681f171'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:24.605165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b353f4aae24694afb146e9b681f171'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:24.608171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-56954662-bdb8-42da-bc1f-b923db645bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b353f4aae24694afb146e9b681f171'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:24.696198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 857/1724] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:23 2022] DELETE /identity/v3/users/a0b353f4aae24694afb146e9b681f171 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:25.690304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:25.691754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:25.728077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187caa6-d176-40a8-9351-ab59024fc554 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:25.877935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:25.926452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:25.930274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:25.930587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:25.932516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] PATH_INFO: `/v3/projects/a4ba5a78562b439ebb0667c0f09c2507` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:25.935505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ba5a78562b439ebb0667c0f09c2507)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:25.942023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:26.183624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf9d3-02ec-4836-80ac-64a1baabd317 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:26.184191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf9d3-02ec-4836-80ac-64a1baabd317 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:26.184641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf9d3-02ec-4836-80ac-64a1baabd317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:26.225475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ba5a78562b439ebb0667c0f09c2507', 'request_id': 'req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:26.234633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5fbdf9d3-02ec-4836-80ac-64a1baabd317 None None] MFA Rules not processed for user `7b1c15ff53e5442881b56bf504b9a31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:26.236149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5fbdf9d3-02ec-4836-80ac-64a1baabd317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:26.312576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ba5a78562b439ebb0667c0f09c2507', 'request_id': 'req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:26.321400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ba5a78562b439ebb0667c0f09c2507', 'request_id': 'req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:26.346636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] The token cache is being invalidate because project a4ba5a78562b439ebb0667c0f09c2507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:26.346946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:26.348897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3aba5e9a-ff05-41a8-97d8-7d77033ecd4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:26.359092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 868/1725] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:25 2022] DELETE /identity/v3/projects/a4ba5a78562b439ebb0667c0f09c2507 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:26.360883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ac5bf-fb32-426b-8a02-8cf78a7ab9b2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:26.361173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ac5bf-fb32-426b-8a02-8cf78a7ab9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:26.361459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ac5bf-fb32-426b-8a02-8cf78a7ab9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:26.427398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-059ac5bf-fb32-426b-8a02-8cf78a7ab9b2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:26.434152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-059ac5bf-fb32-426b-8a02-8cf78a7ab9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:26.434360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 858/1726] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:26.526049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 869/1727] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:26.534247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58e7c7-825a-4687-97c5-553a178f15fd None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:26.534529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58e7c7-825a-4687-97c5-553a178f15fd None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:26.534807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58e7c7-825a-4687-97c5-553a178f15fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:26.581591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6a58e7c7-825a-4687-97c5-553a178f15fd None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:26.586793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6a58e7c7-825a-4687-97c5-553a178f15fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:26.609043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 859/1728] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:26.613174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ac5bf-fb32-426b-8a02-8cf78a7ab9b2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:26.625469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ac5bf-fb32-426b-8a02-8cf78a7ab9b2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:26.630302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:26.630961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:26.631875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:26.632156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:26.632819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:26.633127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:26.634873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:26.649283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 870/1729] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:05:26.653881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58e7c7-825a-4687-97c5-553a178f15fd None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:26.667270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58e7c7-825a-4687-97c5-553a178f15fd None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:26.671849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:26.672473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:26.672789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:26.673071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:26.673723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:26.675847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:26.914179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 860/1730] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:26 2022] POST /identity/v3/projects => generated 379 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:26.919775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:26.938905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bd544-7b1b-4523-9ff6-28a80adcdbad admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:27.104108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:27.104896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:27.105232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:27.105564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:27.106308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:27.108306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:27.245297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 871/1731] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:26 2022] POST /identity/v3/users => generated 290 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:27.249646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:27.277897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a592b-bd76-4b2a-b678-8f826f623852 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:27.300933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:27.310639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:27.311253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:27.311699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:27.312537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:27.324501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:27.360111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 861/1732] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:27.361606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:27.371375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:27.391403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3195-0480-4f16-8351-4d21cfe86c3d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:27.402117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcb997-7e9a-4b28-af77-24b2797367e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:27.403000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb997-7e9a-4b28-af77-24b2797367e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:27.403457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb997-7e9a-4b28-af77-24b2797367e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:27.403901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcb997-7e9a-4b28-af77-24b2797367e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:27.404747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcb997-7e9a-4b28-af77-24b2797367e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:27.406881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcb997-7e9a-4b28-af77-24b2797367e8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:27.436743 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 872/1733] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:27.440771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:27.469717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c948663-b567-460d-a0b9-e2a654b0a1c9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:27.475865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:27.476829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:27.477335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:27.477834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] PATH_INFO: `/v3/projects/333a6f2cf8c34d4e9633d2de84f73b91/users/133ad780054b4096a87ebcc27c8e34b1/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:27.534495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=333a6f2cf8c34d4e9633d2de84f73b91, user_id=133ad780054b4096a87ebcc27c8e34b1, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:27.539377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:27.624440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 862/1734] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:27 2022] PUT /identity/v3/projects/333a6f2cf8c34d4e9633d2de84f73b91/users/133ad780054b4096a87ebcc27c8e34b1/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:27.639901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794f280-6e51-4e88-9ab2-d8e781437803 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:27.639901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794f280-6e51-4e88-9ab2-d8e781437803 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:27.639901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794f280-6e51-4e88-9ab2-d8e781437803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:27.684554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b794f280-6e51-4e88-9ab2-d8e781437803 None None] MFA Rules not processed for user `133ad780054b4096a87ebcc27c8e34b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:27.702591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b794f280-6e51-4e88-9ab2-d8e781437803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:27.845363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:27.846018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 873/1735] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:27.883051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:27.938272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02746dcf-2222-4b36-86b6-bfeb35e12a3d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:27.946980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:27.954726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:27.954726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:27.954726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:27.954726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:27.954726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:28.131682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 863/1736] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:27 2022] POST /identity/v3/projects => generated 377 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:28.136519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794f280-6e51-4e88-9ab2-d8e781437803 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:28.156446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794f280-6e51-4e88-9ab2-d8e781437803 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:28.260736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:28.261444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:28.261918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:28.262200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:28.262893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:28.265588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:28.420372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 874/1737] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:28 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:28.421924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:28.449539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756966e9-a7b1-46a9-b94a-12d17a662e54 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:28.466582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:28.468431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:28.468729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:28.470483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:28.471188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:28.477810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:28.522584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 864/1738] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:28.526689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:28.545529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32528c33-0319-4a70-ab4b-276257c906e9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:28.577867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858e598-1037-49f3-ac4f-6bc5c54a4a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:28.579829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858e598-1037-49f3-ac4f-6bc5c54a4a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:28.580151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858e598-1037-49f3-ac4f-6bc5c54a4a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:28.580440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858e598-1037-49f3-ac4f-6bc5c54a4a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:28.581111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858e598-1037-49f3-ac4f-6bc5c54a4a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:28.590337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858e598-1037-49f3-ac4f-6bc5c54a4a41 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:28.634037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 875/1739] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:28.636718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:28.656805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef724c-7f60-4586-86fb-a4ef4ad14785 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:28.670428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:28.672134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:28.672420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:28.672697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] PATH_INFO: `/v3/projects/ecd232499e92413e99e61cb9066670ee/users/d093bf27342c4d90931b05c4b18cc207/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:28.717905 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd232499e92413e99e61cb9066670ee, user_id=d093bf27342c4d90931b05c4b18cc207, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:28.724288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:28.835407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 865/1740] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:28 2022] PUT /identity/v3/projects/ecd232499e92413e99e61cb9066670ee/users/d093bf27342c4d90931b05c4b18cc207/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:28.859031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008c7e-0588-4c71-9b15-17330b8c4cf8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:28.859643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008c7e-0588-4c71-9b15-17330b8c4cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:28.860261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008c7e-0588-4c71-9b15-17330b8c4cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:28.977258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0d008c7e-0588-4c71-9b15-17330b8c4cf8 None None] MFA Rules not processed for user `d093bf27342c4d90931b05c4b18cc207`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:28.990808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0d008c7e-0588-4c71-9b15-17330b8c4cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:29.120832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:29.157135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 876/1741] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:29.161425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:29.273862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1a91-1f9d-4952-83df-1195bea760b1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:29.284346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:29.284346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:29.284346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:29.284346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:29.284740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:29.284740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:29.834619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 866/1742] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:29 2022] POST /identity/v3/projects => generated 379 bytes in 674 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:29.838757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008c7e-0588-4c71-9b15-17330b8c4cf8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:29.850238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008c7e-0588-4c71-9b15-17330b8c4cf8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:29.923074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:29.923527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:29.923808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:29.924089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:29.924719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:29.927268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:30.084714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 877/1743] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:29 2022] POST /identity/v3/users => generated 296 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:30.084861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:30.097277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47776d8f-22fb-4a3e-ab3c-9a29a178b4cc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:30.109023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:30.109649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:30.109969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:30.110247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:30.111095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:30.112794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:30.137654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 867/1744] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:30.141661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:30.159755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82610557-3e9e-4058-a5bf-17ef6bfb89a4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:30.166692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:30.167498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:30.167849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:30.168160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] PATH_INFO: `/v3/projects/9dc35a3c96034971bc0f6a7116dbce0a/users/9086f5216d9b418fb0cc0b8f7f768e2e/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:30.188359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc35a3c96034971bc0f6a7116dbce0a, user_id=9086f5216d9b418fb0cc0b8f7f768e2e, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:30.194009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:30.291466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 878/1745] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:30 2022] PUT /identity/v3/projects/9dc35a3c96034971bc0f6a7116dbce0a/users/9086f5216d9b418fb0cc0b8f7f768e2e/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:30.295340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:30.313518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06f30b-9ef2-49fa-98fe-2b6f840f0ec6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:30.414679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:30.439675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880cfd-2313-4d6f-9fd6-777997b03f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:30.444291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880cfd-2313-4d6f-9fd6-777997b03f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:30.444291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880cfd-2313-4d6f-9fd6-777997b03f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:30.444291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880cfd-2313-4d6f-9fd6-777997b03f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:30.444291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880cfd-2313-4d6f-9fd6-777997b03f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:30.444291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880cfd-2313-4d6f-9fd6-777997b03f6b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:30.454943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 868/1746] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:30.459253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:30.470440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bded6-8582-4005-a30a-f409ddcb6b81 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:30.475011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:30.475689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:30.475967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:30.476804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] PATH_INFO: `/v3/projects/9dc35a3c96034971bc0f6a7116dbce0a/users/9086f5216d9b418fb0cc0b8f7f768e2e/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:30.478381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc35a3c96034971bc0f6a7116dbce0a, user_id=9086f5216d9b418fb0cc0b8f7f768e2e, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:30.483262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:30.506694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 879/1747] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:30 2022] PUT /identity/v3/projects/9dc35a3c96034971bc0f6a7116dbce0a/users/9086f5216d9b418fb0cc0b8f7f768e2e/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:30.509711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:30.510919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:30.511205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:30.554315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] MFA Rules not processed for user `9086f5216d9b418fb0cc0b8f7f768e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:30.556696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:30.645896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 869/1748] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:30.657672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:30.659431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:30.722497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5a2b4-f493-41b9-a736-144f90ea6bd5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:30.727850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4422cf3-dcf7-4ee3-b9b8-a431bf5587e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:30.728688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4422cf3-dcf7-4ee3-b9b8-a431bf5587e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:30.728891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4422cf3-dcf7-4ee3-b9b8-a431bf5587e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:30.729434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4422cf3-dcf7-4ee3-b9b8-a431bf5587e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:30.744708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4422cf3-dcf7-4ee3-b9b8-a431bf5587e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:30.744708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4422cf3-dcf7-4ee3-b9b8-a431bf5587e8 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:30.755809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 880/1749] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:30.889868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:30.891443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:30.904331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df53ac-1675-4261-a877-adb5d0c41e59 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:30.907990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae1f6d-17f1-4a54-a259-a4ca2228a232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:30.908603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae1f6d-17f1-4a54-a259-a4ca2228a232 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:30.908883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae1f6d-17f1-4a54-a259-a4ca2228a232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:30.909167 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae1f6d-17f1-4a54-a259-a4ca2228a232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:30.970388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae1f6d-17f1-4a54-a259-a4ca2228a232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:30.971843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae1f6d-17f1-4a54-a259-a4ca2228a232 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:30.982484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 870/1750] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:36.438977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8307cce-ac2d-4227-97f1-3f061fe11393 None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:36.439508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8307cce-ac2d-4227-97f1-3f061fe11393 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:36.439508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8307cce-ac2d-4227-97f1-3f061fe11393 None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:36.443020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 881/1751] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:05:36.467788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ff506-4ccd-405b-aa98-d09e4e71fd4e None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:36.468293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ff506-4ccd-405b-aa98-d09e4e71fd4e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:36.468382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ff506-4ccd-405b-aa98-d09e4e71fd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:36.504887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-fb6ff506-4ccd-405b-aa98-d09e4e71fd4e None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:36.518256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fb6ff506-4ccd-405b-aa98-d09e4e71fd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:36.613776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:36.639422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 871/1752] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:05:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:36.659307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8307cce-ac2d-4227-97f1-3f061fe11393 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:36.671060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a8307cce-ac2d-4227-97f1-3f061fe11393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:36.673200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3ce6-bccc-4967-ad6e-fba2c0aafd58 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:36.673482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3ce6-bccc-4967-ad6e-fba2c0aafd58 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:36.673776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3ce6-bccc-4967-ad6e-fba2c0aafd58 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:36.675254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 872/1753] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:05:36.705729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4e59b-e58a-462d-a669-e70b62d3466b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:36.705729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4e59b-e58a-462d-a669-e70b62d3466b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:36.705729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4e59b-e58a-462d-a669-e70b62d3466b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:36.708422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8307cce-ac2d-4227-97f1-3f061fe11393 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:36.715463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:36.716456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:36.716898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:36.717312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:36.727136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-52b4e59b-e58a-462d-a669-e70b62d3466b None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:36.728175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:36.729056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-52b4e59b-e58a-462d-a669-e70b62d3466b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:36.730473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:36.739760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 873/1754] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:05:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:36.743175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 882/1755] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:36.761708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:36.764241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:36.792167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab2809-d00a-4e49-8e88-d653d1ace410 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:36.813570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75f66a-f55b-4764-8a84-ef397e74c54e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:36.813707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f66a-f55b-4764-8a84-ef397e74c54e service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:36.813920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f66a-f55b-4764-8a84-ef397e74c54e service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:36.814243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75f66a-f55b-4764-8a84-ef397e74c54e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:36.845390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75f66a-f55b-4764-8a84-ef397e74c54e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:36.847279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75f66a-f55b-4764-8a84-ef397e74c54e service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:36.867178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 883/1756] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:39.307371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4e59b-e58a-462d-a669-e70b62d3466b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:39.384043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4e59b-e58a-462d-a669-e70b62d3466b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:39.389021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:39.389902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:39.390910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:39.391203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] PATH_INFO: `/v3/users/133ad780054b4096a87ebcc27c8e34b1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:39.392408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=133ad780054b4096a87ebcc27c8e34b1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:39.395069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:39.583157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133ad780054b4096a87ebcc27c8e34b1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:39.583816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133ad780054b4096a87ebcc27c8e34b1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:39.591274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133ad780054b4096a87ebcc27c8e34b1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:39.604032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133ad780054b4096a87ebcc27c8e34b1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:39.604746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133ad780054b4096a87ebcc27c8e34b1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:39.669296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 874/1757] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:39 2022] DELETE /identity/v3/users/133ad780054b4096a87ebcc27c8e34b1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:39.675945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:39.676479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:39.676479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:39.703747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:39.703747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:39.714010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:39.767454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 884/1758] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:39.795837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:39.798025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:39.872231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82ba0-8e17-477c-9b3c-7e1b49af9bcb admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:39.875989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:39.876558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:39.876837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:39.877112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:39.886767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:39.888233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:39.898761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 875/1759] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:40.503082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:40.505196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:40.519358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f799a3a-0d68-45ec-a53e-701c22259387 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:40.527114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:40.528083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:40.528083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:40.528771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] PATH_INFO: `/v3/projects/333a6f2cf8c34d4e9633d2de84f73b91` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:40.530015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] RBAC: Authorizing `identity:delete_project(project_id=333a6f2cf8c34d4e9633d2de84f73b91)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:40.532151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:40.678777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333a6f2cf8c34d4e9633d2de84f73b91', 'request_id': 'req-83786622-16a3-475e-b2ec-bf402058d35c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:40.757651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333a6f2cf8c34d4e9633d2de84f73b91', 'request_id': 'req-83786622-16a3-475e-b2ec-bf402058d35c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:40.764951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333a6f2cf8c34d4e9633d2de84f73b91', 'request_id': 'req-83786622-16a3-475e-b2ec-bf402058d35c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:40.776469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] The token cache is being invalidate because project 333a6f2cf8c34d4e9633d2de84f73b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:40.776840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:40.777506 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:40.779342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 885/1760] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:40 2022] DELETE /identity/v3/projects/333a6f2cf8c34d4e9633d2de84f73b91 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:40.782980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:40.793729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f3715-d9cb-4597-830c-881944df1a7d service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:40.802061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:40.862342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:40.863411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:40.863698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:40.863972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] PATH_INFO: `/v3/users/d093bf27342c4d90931b05c4b18cc207` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:40.865165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d093bf27342c4d90931b05c4b18cc207)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:40.867420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:41.070736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd093bf27342c4d90931b05c4b18cc207'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.071755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd093bf27342c4d90931b05c4b18cc207'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.079616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd093bf27342c4d90931b05c4b18cc207'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.092543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd093bf27342c4d90931b05c4b18cc207'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.093880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd093bf27342c4d90931b05c4b18cc207'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.166062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 876/1761] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:40 2022] DELETE /identity/v3/users/d093bf27342c4d90931b05c4b18cc207 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:41.571512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:41.573166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:41.590341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83786622-16a3-475e-b2ec-bf402058d35c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:41.667659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:41.668485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:41.668828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:41.669169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] PATH_INFO: `/v3/projects/ecd232499e92413e99e61cb9066670ee` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:41.671059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd232499e92413e99e61cb9066670ee)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:41.673035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:41.772474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:41.778859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd232499e92413e99e61cb9066670ee', 'request_id': 'req-e25bbda6-3eea-462c-9536-ce647fddcbf0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.860865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd232499e92413e99e61cb9066670ee', 'request_id': 'req-e25bbda6-3eea-462c-9536-ce647fddcbf0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.869466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd232499e92413e99e61cb9066670ee', 'request_id': 'req-e25bbda6-3eea-462c-9536-ce647fddcbf0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.880026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] The token cache is being invalidate because project ecd232499e92413e99e61cb9066670ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:41.880312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.880959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:41.882794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 886/1762] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:41 2022] DELETE /identity/v3/projects/ecd232499e92413e99e61cb9066670ee => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:41.886627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:41.888563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:41.901353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc78c7-7132-423e-84c5-5bea32c64bc4 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:41.959537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:41.960178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:41.960463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:41.960734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] PATH_INFO: `/v3/users/9086f5216d9b418fb0cc0b8f7f768e2e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:41.961885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9086f5216d9b418fb0cc0b8f7f768e2e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:41.963851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:42.149622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9086f5216d9b418fb0cc0b8f7f768e2e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:42.150547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9086f5216d9b418fb0cc0b8f7f768e2e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:42.173122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9086f5216d9b418fb0cc0b8f7f768e2e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:42.203805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9086f5216d9b418fb0cc0b8f7f768e2e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:42.204536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-9a9bd114-a75c-4bf5-915a-b69fbbd5dfa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9086f5216d9b418fb0cc0b8f7f768e2e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:42.321729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 877/1763] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:41 2022] DELETE /identity/v3/users/9086f5216d9b418fb0cc0b8f7f768e2e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:42.396180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:43.839970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:43.841715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:43.869541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bbda6-3eea-462c-9536-ce647fddcbf0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:43.974562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:43.975520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:43.975859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:43.976142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] PATH_INFO: `/v3/projects/9dc35a3c96034971bc0f6a7116dbce0a` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:43.977470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc35a3c96034971bc0f6a7116dbce0a)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:43.980114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:44.195764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc35a3c96034971bc0f6a7116dbce0a', 'request_id': 'req-ea23f790-795e-4c76-a0de-fb755912dabe'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:44.301971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc35a3c96034971bc0f6a7116dbce0a', 'request_id': 'req-ea23f790-795e-4c76-a0de-fb755912dabe'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:44.313936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc35a3c96034971bc0f6a7116dbce0a', 'request_id': 'req-ea23f790-795e-4c76-a0de-fb755912dabe'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:44.327691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] The token cache is being invalidate because project 9dc35a3c96034971bc0f6a7116dbce0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:44.327985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:44.328666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ea23f790-795e-4c76-a0de-fb755912dabe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:44.330593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 887/1764] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:43 2022] DELETE /identity/v3/projects/9dc35a3c96034971bc0f6a7116dbce0a => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:44.336930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882142-28d9-4caa-82ec-8ca827a6cb71 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:44.337484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882142-28d9-4caa-82ec-8ca827a6cb71 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:44.337484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb882142-28d9-4caa-82ec-8ca827a6cb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:44.364813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-eb882142-28d9-4caa-82ec-8ca827a6cb71 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:44.366673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-eb882142-28d9-4caa-82ec-8ca827a6cb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:44.430944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 878/1765] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:44.439327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0224c-e405-49ba-9e07-373cf3dbc856 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:44.439837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0224c-e405-49ba-9e07-373cf3dbc856 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:44.440269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0224c-e405-49ba-9e07-373cf3dbc856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:44.465526 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a9c0224c-e405-49ba-9e07-373cf3dbc856 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:44.466874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a9c0224c-e405-49ba-9e07-373cf3dbc856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:44.513113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 888/1766] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:44.525344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882142-28d9-4caa-82ec-8ca827a6cb71 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:44.538969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb882142-28d9-4caa-82ec-8ca827a6cb71 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:44.557409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:44.558968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:44.559267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:44.560447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:44.561170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:44.561438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:44.563088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:44.577396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 879/1767] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:05:44.582938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0224c-e405-49ba-9e07-373cf3dbc856 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:44.594747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0224c-e405-49ba-9e07-373cf3dbc856 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:44.601778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:44.602187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:44.603082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:44.603082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:44.603778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:44.606232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:44.752415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 889/1768] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:44 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:44.757308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:44.769502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bdf0c-c426-46d1-ad8f-ca1b20a9e1a5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:44.842788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:44.843438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:44.843712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:44.843983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:44.844620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:44.846562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:44.973591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 880/1769] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:44 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:44.978319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:45.021511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75425cc-44b1-460e-b69b-15d78b23ef37 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:45.030065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:45.031270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:45.031559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:45.043730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:45.043730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:45.043730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:45.050929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 890/1770] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:45.052954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:45.067210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54b49b-7e6d-4780-94a2-ab2c477fedcb admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:45.072793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7c13c-3156-449c-aed8-d14604904759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:45.073412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7c13c-3156-449c-aed8-d14604904759 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:45.073727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7c13c-3156-449c-aed8-d14604904759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:45.074348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7c13c-3156-449c-aed8-d14604904759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:45.075162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7c13c-3156-449c-aed8-d14604904759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:45.077075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7c13c-3156-449c-aed8-d14604904759 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:45.096351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 881/1771] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:05:45.100351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:45.112773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de860db8-f47d-4cd7-b21e-5d05a3719971 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:45.119029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:45.119947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:45.120461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:45.120932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] PATH_INFO: `/v3/projects/24a1fa3df8ab41338c41feab0bad75e3/users/41bab1f30922408d9769b6f8d0f7fa6e/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:45.141978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a1fa3df8ab41338c41feab0bad75e3, user_id=41bab1f30922408d9769b6f8d0f7fa6e, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:45.144134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:45.204631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:45.216564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 891/1772] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:45 2022] PUT /identity/v3/projects/24a1fa3df8ab41338c41feab0bad75e3/users/41bab1f30922408d9769b6f8d0f7fa6e/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:45.235654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640d6ba-68a5-4966-bfcc-c78041c0ee47 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:45.236458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640d6ba-68a5-4966-bfcc-c78041c0ee47 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:45.236912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640d6ba-68a5-4966-bfcc-c78041c0ee47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:45.279955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0640d6ba-68a5-4966-bfcc-c78041c0ee47 None None] MFA Rules not processed for user `41bab1f30922408d9769b6f8d0f7fa6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:45.281891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0640d6ba-68a5-4966-bfcc-c78041c0ee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:45.290649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:45.418993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 882/1773] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:45.432363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:45.434105 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:45.501018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e55be-ac70-44eb-ae3d-0ab8b573dec0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:45.508580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797f381-a689-4845-b798-77dddd202de8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:45.509269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797f381-a689-4845-b798-77dddd202de8 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:45.511616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797f381-a689-4845-b798-77dddd202de8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:45.511616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797f381-a689-4845-b798-77dddd202de8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:45.519674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797f381-a689-4845-b798-77dddd202de8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:45.521303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797f381-a689-4845-b798-77dddd202de8 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:45.531488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 892/1774] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:45.555220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640d6ba-68a5-4966-bfcc-c78041c0ee47 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:45.618953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640d6ba-68a5-4966-bfcc-c78041c0ee47 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:45.623849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:45.624656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:45.625032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:45.625403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] PATH_INFO: `/v3/users/41bab1f30922408d9769b6f8d0f7fa6e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:45.626678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bab1f30922408d9769b6f8d0f7fa6e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:45.628876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:45.800421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bab1f30922408d9769b6f8d0f7fa6e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:45.801095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bab1f30922408d9769b6f8d0f7fa6e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:45.809119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bab1f30922408d9769b6f8d0f7fa6e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:45.820962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bab1f30922408d9769b6f8d0f7fa6e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:45.821682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bab1f30922408d9769b6f8d0f7fa6e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:45.895136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 883/1775] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:45 2022] DELETE /identity/v3/users/41bab1f30922408d9769b6f8d0f7fa6e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:45.900211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:45.900543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:45.900788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:45.920074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:45.921603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:45.980011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 893/1776] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:45.989879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:45.991836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:46.059179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044748ea-f841-4dd9-9f76-99948b10d999 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:46.065454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a512ea-4e69-407e-a613-eac1e64163dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:46.066394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a512ea-4e69-407e-a613-eac1e64163dc service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:46.066677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a512ea-4e69-407e-a613-eac1e64163dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:46.067100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a512ea-4e69-407e-a613-eac1e64163dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:46.077486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a512ea-4e69-407e-a613-eac1e64163dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:46.078927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a512ea-4e69-407e-a613-eac1e64163dc service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:46.090418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 884/1777] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:46.587896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:46.589692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:46.603982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc01b8f-4b91-41c6-a858-a831395e613e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:46.609172 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:46.609828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:46.610324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:46.610791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] PATH_INFO: `/v3/projects/24a1fa3df8ab41338c41feab0bad75e3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:46.611946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a1fa3df8ab41338c41feab0bad75e3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:46.613921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:46.649651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:46.753764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a1fa3df8ab41338c41feab0bad75e3', 'request_id': 'req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:46.832455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a1fa3df8ab41338c41feab0bad75e3', 'request_id': 'req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:46.840623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a1fa3df8ab41338c41feab0bad75e3', 'request_id': 'req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:46.852219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] The token cache is being invalidate because project 24a1fa3df8ab41338c41feab0bad75e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:46.852513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:46.853302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d58a3ed7-bfa7-49d7-85a4-e0f08eae62a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:46.855143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 894/1778] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:46 2022] DELETE /identity/v3/projects/24a1fa3df8ab41338c41feab0bad75e3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:46.859856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485c5a1-38c2-46ad-a218-7dbe04282930 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:46.860144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485c5a1-38c2-46ad-a218-7dbe04282930 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:46.860421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485c5a1-38c2-46ad-a218-7dbe04282930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:46.889767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-c485c5a1-38c2-46ad-a218-7dbe04282930 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:46.891128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c485c5a1-38c2-46ad-a218-7dbe04282930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:46.971735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 885/1779] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:46.984409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59981743-c5e1-4efc-aadd-e526a7f22d14 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:46.985000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59981743-c5e1-4efc-aadd-e526a7f22d14 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:46.985000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59981743-c5e1-4efc-aadd-e526a7f22d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.042633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-59981743-c5e1-4efc-aadd-e526a7f22d14 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:47.044267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-59981743-c5e1-4efc-aadd-e526a7f22d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:47.062572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 895/1780] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:47.066688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485c5a1-38c2-46ad-a218-7dbe04282930 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.079888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485c5a1-38c2-46ad-a218-7dbe04282930 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.089297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.090396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.091036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.091506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.092436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:47.092888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.107672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:47.136164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 886/1781] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:05:47.137311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59981743-c5e1-4efc-aadd-e526a7f22d14 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.148786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59981743-c5e1-4efc-aadd-e526a7f22d14 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.155629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.156507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.157365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.157365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.158061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.160555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:47.189602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:47.261091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 896/1782] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:47 2022] POST /identity/v3/projects => generated 361 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:47.264507 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.275827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed741ebd-fd89-4438-b730-854f369ffe8f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.331785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.332386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.332712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.332931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.333569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.336174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:47.397584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 887/1783] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:47 2022] POST /identity/v3/users => generated 281 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:47.401998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.412319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336cbf-4fd7-4ac2-bc82-36ad39f32c92 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.417454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.418093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.418376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.418692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.419375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.421231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:47.432236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 897/1784] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:47.436168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.447005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58f3c-5327-4c8a-9bf8-d8a62c6f3f93 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.451769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97018a1a-fdc7-4d37-9fb2-33efd5b99f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.452387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97018a1a-fdc7-4d37-9fb2-33efd5b99f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.452669 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97018a1a-fdc7-4d37-9fb2-33efd5b99f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.452940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97018a1a-fdc7-4d37-9fb2-33efd5b99f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.453583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97018a1a-fdc7-4d37-9fb2-33efd5b99f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.455226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97018a1a-fdc7-4d37-9fb2-33efd5b99f68 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:47.466861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 888/1785] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:47.471330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.481191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b620e9-d343-49a4-8ee5-57f7e8187660 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.485763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.486408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.486690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.486968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] PATH_INFO: `/v3/projects/95073ab7737447c0b011a7de3aa734d1/users/e189f5c09bb546e792d7e91a116a4212/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.502067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95073ab7737447c0b011a7de3aa734d1, user_id=e189f5c09bb546e792d7e91a116a4212, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.504702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:47.592466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 898/1786] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:47 2022] PUT /identity/v3/projects/95073ab7737447c0b011a7de3aa734d1/users/e189f5c09bb546e792d7e91a116a4212/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:47.602880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87f039-5b07-4ddb-bfca-d2c460701729 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.603165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87f039-5b07-4ddb-bfca-d2c460701729 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.603469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87f039-5b07-4ddb-bfca-d2c460701729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.634591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4f87f039-5b07-4ddb-bfca-d2c460701729 None None] MFA Rules not processed for user `e189f5c09bb546e792d7e91a116a4212`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:47.635516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4f87f039-5b07-4ddb-bfca-d2c460701729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:47.659181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:47.704721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 889/1787] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:47.742008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.743694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:47.804594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1b6d7-1448-4de1-9a7c-82297f1c48b7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.809790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0af39-4c0e-4884-81ca-85ec29d46c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.809790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0af39-4c0e-4884-81ca-85ec29d46c8e service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.809790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0af39-4c0e-4884-81ca-85ec29d46c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.809790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0af39-4c0e-4884-81ca-85ec29d46c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.819998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0af39-4c0e-4884-81ca-85ec29d46c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.821437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0af39-4c0e-4884-81ca-85ec29d46c8e service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:47.833212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 899/1788] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:47.849416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87f039-5b07-4ddb-bfca-d2c460701729 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:47.917021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87f039-5b07-4ddb-bfca-d2c460701729 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:47.921899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:47.922535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:47.922815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:47.923078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] PATH_INFO: `/v3/users/e189f5c09bb546e792d7e91a116a4212` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:47.924401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e189f5c09bb546e792d7e91a116a4212)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:47.926371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:48.154284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e189f5c09bb546e792d7e91a116a4212'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:48.155231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e189f5c09bb546e792d7e91a116a4212'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:48.166314 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e189f5c09bb546e792d7e91a116a4212'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:48.179852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e189f5c09bb546e792d7e91a116a4212'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:48.180844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e189f5c09bb546e792d7e91a116a4212'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:48.309178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 890/1789] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:47 2022] DELETE /identity/v3/users/e189f5c09bb546e792d7e91a116a4212 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:48.328422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:48.328887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:48.328887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:48.354702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:48.354702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:48.463314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 900/1790] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:48.477692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:48.479459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:48.557921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3fb17-a700-4792-a59f-e56708160c30 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:48.562597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a1e96-2bf5-42b7-b262-fb1e496e07a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:48.563340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1e96-2bf5-42b7-b262-fb1e496e07a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:48.563749 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1e96-2bf5-42b7-b262-fb1e496e07a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:48.564120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1e96-2bf5-42b7-b262-fb1e496e07a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:48.576284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a1e96-2bf5-42b7-b262-fb1e496e07a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:48.577921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a1e96-2bf5-42b7-b262-fb1e496e07a4 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:48.586104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:48.592003 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 891/1791] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:49.114241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:49.117834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:49.136926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e67a04-d26d-470d-8fa6-06122c635e7c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:49.144700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:49.144700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:49.144700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:49.144700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] PATH_INFO: `/v3/projects/95073ab7737447c0b011a7de3aa734d1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:49.146445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95073ab7737447c0b011a7de3aa734d1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:49.148696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:49.351310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95073ab7737447c0b011a7de3aa734d1', 'request_id': 'req-00d65c9c-23b3-422c-9e57-940f63ae1712'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:49.422566 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95073ab7737447c0b011a7de3aa734d1', 'request_id': 'req-00d65c9c-23b3-422c-9e57-940f63ae1712'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:49.429905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:49.435998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95073ab7737447c0b011a7de3aa734d1', 'request_id': 'req-00d65c9c-23b3-422c-9e57-940f63ae1712'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:49.447470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] The token cache is being invalidate because project 95073ab7737447c0b011a7de3aa734d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:49.448033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:49.449930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-00d65c9c-23b3-422c-9e57-940f63ae1712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:49.451899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 901/1792] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:49 2022] DELETE /identity/v3/projects/95073ab7737447c0b011a7de3aa734d1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:49.456686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef6d5-8b11-48bb-a17a-051014cc773d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:49.457169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef6d5-8b11-48bb-a17a-051014cc773d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:49.457256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef6d5-8b11-48bb-a17a-051014cc773d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:49.488635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-34bef6d5-8b11-48bb-a17a-051014cc773d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:49.490088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-34bef6d5-8b11-48bb-a17a-051014cc773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:49.567011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 892/1793] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:49.576062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc72fd-a421-4257-9b02-47ca4dcae613 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:49.576483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc72fd-a421-4257-9b02-47ca4dcae613 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:49.576859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc72fd-a421-4257-9b02-47ca4dcae613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:49.598464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-74fc72fd-a421-4257-9b02-47ca4dcae613 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:49.600146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-74fc72fd-a421-4257-9b02-47ca4dcae613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:49.608623 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 902/1794] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:49.612933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef6d5-8b11-48bb-a17a-051014cc773d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:49.624432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef6d5-8b11-48bb-a17a-051014cc773d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:49.633932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:49.634642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:49.634982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:49.635263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:49.635975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:49.636269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:49.638010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:49.656518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 893/1795] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:05:49.661494 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc72fd-a421-4257-9b02-47ca4dcae613 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:49.674501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc72fd-a421-4257-9b02-47ca4dcae613 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:49.682046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:49.682740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:49.683082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:49.683415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:49.684333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:49.686729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:49.798947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 903/1796] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:49 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:49.803982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:49.817203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db29f9-c243-404d-b93f-dc7874f21e67 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:49.914927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:49.915775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:49.916177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:49.916621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:49.917239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:49.919468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:50.004017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 894/1797] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:49 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:50.008684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:50.029426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045ba4f-42f9-414f-a08c-7fe2b805deb8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:50.047056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:50.048634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.049066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.049489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.050344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:50.052129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:50.080963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 904/1798] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:50.087020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:50.100096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef22ba6-5977-4a26-883f-c8f4ee5e3c77 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:50.107303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f60234-cda6-477b-8177-65e26b9c395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:50.108026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f60234-cda6-477b-8177-65e26b9c395f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.108310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f60234-cda6-477b-8177-65e26b9c395f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.108585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f60234-cda6-477b-8177-65e26b9c395f admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.109269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f60234-cda6-477b-8177-65e26b9c395f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:50.110929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f60234-cda6-477b-8177-65e26b9c395f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:50.123446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 895/1799] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:50.127857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:50.139678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975e1ca-0287-4831-a628-b2e4447828bb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:50.151386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:50.151386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.151386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.151386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] PATH_INFO: `/v3/projects/0b25544a0b2a4cc19391f0a5c2db9a38/users/acca65806480468697a1a49a090d55de/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.164139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b25544a0b2a4cc19391f0a5c2db9a38, user_id=acca65806480468697a1a49a090d55de, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:50.166951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:50.289981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 905/1800] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:50 2022] PUT /identity/v3/projects/0b25544a0b2a4cc19391f0a5c2db9a38/users/acca65806480468697a1a49a090d55de/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:50.300984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115c18-5af5-403e-9137-f6f8515db0a7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.301268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115c18-5af5-403e-9137-f6f8515db0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.301540 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115c18-5af5-403e-9137-f6f8515db0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.337705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f1115c18-5af5-403e-9137-f6f8515db0a7 None None] MFA Rules not processed for user `acca65806480468697a1a49a090d55de`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:50.339254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f1115c18-5af5-403e-9137-f6f8515db0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:50.384911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:50.418104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 896/1801] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:50.424135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:50.513449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979fbb5-bce6-4a37-ba33-7794757ed098 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:50.518226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:50.518825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.519103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.519374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.520002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:50.522452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:50.629048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 906/1802] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:50.630518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115c18-5af5-403e-9137-f6f8515db0a7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:50.641600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115c18-5af5-403e-9137-f6f8515db0a7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:50.699494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:50.700120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.700411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.700701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.701338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:50.703535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:50.811815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 897/1803] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:50 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:50.812761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:50.826688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27416af1-c954-44f7-a4cf-c5aff01dd6fb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:50.836178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:50.836622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.837106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.837534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.838372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:50.840618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:50.856433 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 907/1804] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:05:50.864881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:50.878608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa42bdc-d13d-4bd3-98a3-c6f177a42e08 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:50.884989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:50.885349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:50.885609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:50.885956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] PATH_INFO: `/v3/projects/22b1c90ff50a4105850f862a43799961/users/725fef9561de4db8abea83a3fdb0501b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:50.913486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b1c90ff50a4105850f862a43799961, user_id=725fef9561de4db8abea83a3fdb0501b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:50.917436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:51.020583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 898/1805] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:50 2022] PUT /identity/v3/projects/22b1c90ff50a4105850f862a43799961/users/725fef9561de4db8abea83a3fdb0501b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:51.025294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:51.039806 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ba7ee-39ae-4477-a865-b242f5f421f3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:51.130192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:51.144991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15e568-a6fb-449d-94a3-cb10fcfefaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:51.145881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15e568-a6fb-449d-94a3-cb10fcfefaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:51.146174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15e568-a6fb-449d-94a3-cb10fcfefaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:51.148084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15e568-a6fb-449d-94a3-cb10fcfefaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:51.148793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15e568-a6fb-449d-94a3-cb10fcfefaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:51.150554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15e568-a6fb-449d-94a3-cb10fcfefaa1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:51.162600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 908/1806] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:05:51.166533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:51.179343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f795a85-6240-4c33-9095-cd0f6acd39d4 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:51.184233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:51.184904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:51.185181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:51.185460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] PATH_INFO: `/v3/projects/22b1c90ff50a4105850f862a43799961/users/725fef9561de4db8abea83a3fdb0501b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:51.187395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b1c90ff50a4105850f862a43799961, user_id=725fef9561de4db8abea83a3fdb0501b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:51.200540 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:51.208006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 899/1807] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:51 2022] PUT /identity/v3/projects/22b1c90ff50a4105850f862a43799961/users/725fef9561de4db8abea83a3fdb0501b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:51.219445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:51.219734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:51.220015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:51.255365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] MFA Rules not processed for user `725fef9561de4db8abea83a3fdb0501b`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:51.257634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:51.331863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 909/1808] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:51.343375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:51.345053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:51.414966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c148961-c301-466d-86d2-aafd685a8871 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:51.418788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6324a7c-16cf-4a18-bbfc-ff38885878fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:51.419423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324a7c-16cf-4a18-bbfc-ff38885878fb service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:51.419720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324a7c-16cf-4a18-bbfc-ff38885878fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:51.419989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324a7c-16cf-4a18-bbfc-ff38885878fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:51.465407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:51.477798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6324a7c-16cf-4a18-bbfc-ff38885878fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:51.479261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6324a7c-16cf-4a18-bbfc-ff38885878fb service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:51.488647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 900/1809] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:53.875450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:53.878192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:53.983206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01078c-70b7-488d-84d4-c294e5e6e6ae None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:53.990276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:53.990926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:53.991205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:53.992796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] PATH_INFO: `/v3/users/715eab47227449cbaa5c00d0fb609647` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:53.995387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=715eab47227449cbaa5c00d0fb609647)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:54.000958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:54.215431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715eab47227449cbaa5c00d0fb609647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:54.216161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715eab47227449cbaa5c00d0fb609647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:54.225517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715eab47227449cbaa5c00d0fb609647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:54.243128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715eab47227449cbaa5c00d0fb609647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:54.244466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715eab47227449cbaa5c00d0fb609647'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:54.336387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:54.342301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 910/1810] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:53 2022] DELETE /identity/v3/users/715eab47227449cbaa5c00d0fb609647 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:54.346726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:54.347229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:54.347318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:54.371345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:54.372594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:54.454872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 901/1811] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:54.468357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:54.470651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:54.532097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7fa1d-a829-4265-b2d0-ee020ae4ddc6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:54.535987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:54.536593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:54.536874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:54.537152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:54.547186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:54.548612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:54.559058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 911/1812] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:54.970398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:54.972419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:54.985247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ce10f-b762-4f44-a180-70b6ae17fb84 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:54.991267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:54.991993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:54.992274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:54.992552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] PATH_INFO: `/v3/users/acca65806480468697a1a49a090d55de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:54.994919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acca65806480468697a1a49a090d55de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:54.996342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:55.119593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:55.121344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:55.137339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700c99-9f4b-4b27-beb4-2c91ffb87ee4 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:55.151724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:55.151724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:55.151724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:55.151724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] PATH_INFO: `/v3/projects/4d0b8e1853944a739c91e03c0d235682` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:55.151987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0b8e1853944a739c91e03c0d235682)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:55.151987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:55.214305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acca65806480468697a1a49a090d55de'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.218140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acca65806480468697a1a49a090d55de'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.226792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acca65806480468697a1a49a090d55de'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.240916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acca65806480468697a1a49a090d55de'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.242358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0b43a185-e853-4fbf-b2c2-f481d6754775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acca65806480468697a1a49a090d55de'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.272620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0b8e1853944a739c91e03c0d235682', 'request_id': 'req-293cc38b-6d30-4d9c-ac85-c84492dac09f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.306822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 902/1813] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:54 2022] DELETE /identity/v3/users/acca65806480468697a1a49a090d55de => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:55.311963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:55.312525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:55.312525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:55.335563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:55.336852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:55.357397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0b8e1853944a739c91e03c0d235682', 'request_id': 'req-293cc38b-6d30-4d9c-ac85-c84492dac09f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.367429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0b8e1853944a739c91e03c0d235682', 'request_id': 'req-293cc38b-6d30-4d9c-ac85-c84492dac09f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.379634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] The token cache is being invalidate because project 4d0b8e1853944a739c91e03c0d235682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:55.379924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.380648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.382761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 912/1814] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:55 2022] DELETE /identity/v3/projects/4d0b8e1853944a739c91e03c0d235682 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:55.386535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:55.387936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:55.400898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:55.410234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 903/1815] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:55.421269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:55.422573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:55.472098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cc38b-6d30-4d9c-ac85-c84492dac09f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:55.477554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:55.478276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:55.478558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:55.478837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] PATH_INFO: `/v3/users/0c8eb43f61404fc28d94c319d3e33d13` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:55.480087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8eb43f61404fc28d94c319d3e33d13)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:55.482108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:55.490572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3393cbe-a144-4f8d-a93d-e352bd6c0e59 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:55.497459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:55.498149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:55.498427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:55.498701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:55.509023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:55.510571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:55.521318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 904/1816] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:05:55.564221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:55.659464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8eb43f61404fc28d94c319d3e33d13'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.660262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8eb43f61404fc28d94c319d3e33d13'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.671049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8eb43f61404fc28d94c319d3e33d13'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.684406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8eb43f61404fc28d94c319d3e33d13'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.685190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8eb43f61404fc28d94c319d3e33d13'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:55.760716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 913/1817] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:55 2022] DELETE /identity/v3/users/0c8eb43f61404fc28d94c319d3e33d13 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:55.999023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:56.000792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:56.030524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebe207-24c0-45c6-85c5-32d00d425503 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:56.113946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:56.114606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:56.114882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:56.115161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] PATH_INFO: `/v3/projects/0b25544a0b2a4cc19391f0a5c2db9a38` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:56.116356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b25544a0b2a4cc19391f0a5c2db9a38)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:56.118373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:56.246304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b25544a0b2a4cc19391f0a5c2db9a38', 'request_id': 'req-32d688cc-7957-4ffe-ac50-e85761133b53'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.334900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b25544a0b2a4cc19391f0a5c2db9a38', 'request_id': 'req-32d688cc-7957-4ffe-ac50-e85761133b53'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.346508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b25544a0b2a4cc19391f0a5c2db9a38', 'request_id': 'req-32d688cc-7957-4ffe-ac50-e85761133b53'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.368464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] The token cache is being invalidate because project 0b25544a0b2a4cc19391f0a5c2db9a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:56.368970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.369677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.371236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 905/1818] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:55 2022] DELETE /identity/v3/projects/0b25544a0b2a4cc19391f0a5c2db9a38 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:56.377869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:56.377869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:56.390162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55973b8a-047b-4351-a5cd-79199fff45af admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:56.437822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:56.439312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:56.488544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:56.489545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:56.489977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:56.491405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:56.491853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] PATH_INFO: `/v3/users/725fef9561de4db8abea83a3fdb0501b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:56.493438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725fef9561de4db8abea83a3fdb0501b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:56.495729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:56.534627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d688cc-7957-4ffe-ac50-e85761133b53 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:56.546208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:56.546208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:56.546208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:56.546208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] PATH_INFO: `/v3/projects/7e2e2b414acd4accba56b193c2fc3952` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:56.546620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2e2b414acd4accba56b193c2fc3952)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:56.547823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:56.703108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2e2b414acd4accba56b193c2fc3952', 'request_id': 'req-bf1e999e-3dfa-4525-ae24-57befeb30f88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.703391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725fef9561de4db8abea83a3fdb0501b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.704190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725fef9561de4db8abea83a3fdb0501b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.712076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725fef9561de4db8abea83a3fdb0501b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.724051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725fef9561de4db8abea83a3fdb0501b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.724754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f3d0fe66-e0ab-4aad-ac4a-a1b8269a9355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725fef9561de4db8abea83a3fdb0501b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.773095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2e2b414acd4accba56b193c2fc3952', 'request_id': 'req-bf1e999e-3dfa-4525-ae24-57befeb30f88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.780163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2e2b414acd4accba56b193c2fc3952', 'request_id': 'req-bf1e999e-3dfa-4525-ae24-57befeb30f88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.791367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] The token cache is being invalidate because project 7e2e2b414acd4accba56b193c2fc3952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:56.791658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.792324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf1e999e-3dfa-4525-ae24-57befeb30f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:56.793880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 906/1819] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:56 2022] DELETE /identity/v3/projects/7e2e2b414acd4accba56b193c2fc3952 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:56.798704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8641e47-b78a-499f-8376-4ee65916df35 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:56.798992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8641e47-b78a-499f-8376-4ee65916df35 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:56.799267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8641e47-b78a-499f-8376-4ee65916df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:56.819500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-c8641e47-b78a-499f-8376-4ee65916df35 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:56.820692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c8641e47-b78a-499f-8376-4ee65916df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:56.831533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 914/1820] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:05:56 2022] DELETE /identity/v3/users/725fef9561de4db8abea83a3fdb0501b => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:56.884186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 907/1821] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:56.891603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d83f6-e031-40dc-8621-d7186ad56c10 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:56.891910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d83f6-e031-40dc-8621-d7186ad56c10 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:56.892537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d83f6-e031-40dc-8621-d7186ad56c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:56.917473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-745d83f6-e031-40dc-8621-d7186ad56c10 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:56.919055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-745d83f6-e031-40dc-8621-d7186ad56c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:56.938710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:56.980751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 915/1822] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:56.985057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8641e47-b78a-499f-8376-4ee65916df35 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:56.995612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8641e47-b78a-499f-8376-4ee65916df35 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.000262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.000874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.001156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.005297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.005297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:57.005297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.005297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.015886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 908/1823] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:05:57.020254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d83f6-e031-40dc-8621-d7186ad56c10 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.030297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d83f6-e031-40dc-8621-d7186ad56c10 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.036890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.037909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.045899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.045899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.045899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.045899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.137838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 916/1824] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:57.141666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.152599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175ca92-2533-4b53-a13a-81628b8aaf7a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.203685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:57.214659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.215512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.215886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.216252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.217229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.219367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.284382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.286134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:57.301262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5799e-9b4a-4dc9-be6f-030221b74cc6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.307033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.307677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.307958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.308232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] PATH_INFO: `/v3/projects/22b1c90ff50a4105850f862a43799961` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.309357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b1c90ff50a4105850f862a43799961)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.311471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.318647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 909/1825] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:57.321195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.336078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca08c42-2711-4ec8-8f15-a1b7393d272d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.343633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.344253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.344537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.344802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.345450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.347143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.358645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 910/1826] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:57.362700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.373422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066888-1ee9-4057-9550-22acd5d3addb admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.380776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.381381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.381658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.382063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.382900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.384489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.394900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 911/1827] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:57.399389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.411769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f7358-2dfb-4c65-aa10-f94df9faf75f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.417945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20c1df-66e1-4b14-b632-22e90c99c94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.418618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20c1df-66e1-4b14-b632-22e90c99c94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.418900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20c1df-66e1-4b14-b632-22e90c99c94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.419179 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20c1df-66e1-4b14-b632-22e90c99c94c admin admin] PATH_INFO: `/v3/projects/dc3f24b64c064b849e6dcda9b9871019/users/d8097c27429e4e778d47e725c9aaa083/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.430421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b1c90ff50a4105850f862a43799961', 'request_id': 'req-1ac0c875-bfdf-4f5e-9dec-33d980366804'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:57.435185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20c1df-66e1-4b14-b632-22e90c99c94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3f24b64c064b849e6dcda9b9871019, user_id=d8097c27429e4e778d47e725c9aaa083, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.437914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20c1df-66e1-4b14-b632-22e90c99c94c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.510323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b1c90ff50a4105850f862a43799961', 'request_id': 'req-1ac0c875-bfdf-4f5e-9dec-33d980366804'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:57.518000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b1c90ff50a4105850f862a43799961', 'request_id': 'req-1ac0c875-bfdf-4f5e-9dec-33d980366804'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:57.519983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 912/1828] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:57 2022] PUT /identity/v3/projects/dc3f24b64c064b849e6dcda9b9871019/users/d8097c27429e4e778d47e725c9aaa083/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:57.531731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.532067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.532319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.552180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] The token cache is being invalidate because project 22b1c90ff50a4105850f862a43799961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:05:57.552484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:57.555795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1ac0c875-bfdf-4f5e-9dec-33d980366804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:05:57.558596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 917/1829] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:05:57 2022] DELETE /identity/v3/projects/22b1c90ff50a4105850f862a43799961 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:57.565617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43a6f-8af0-4356-ab80-4b023eed5807 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.565962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43a6f-8af0-4356-ab80-4b023eed5807 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.566254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f43a6f-8af0-4356-ab80-4b023eed5807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.574341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] MFA Rules not processed for user `d8097c27429e4e778d47e725c9aaa083`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:57.575725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:57.590110 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d9f43a6f-8af0-4356-ab80-4b023eed5807 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:57.591356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d9f43a6f-8af0-4356-ab80-4b023eed5807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:57.678153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 913/1830] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:57.682634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.684865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:57.698045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 918/1831] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:57.706038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625118b-0940-490b-adba-7eb927f42ace None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.706399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625118b-0940-490b-adba-7eb927f42ace None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.707267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625118b-0940-490b-adba-7eb927f42ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.714157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dcd33-8aa6-4798-844f-db0ed311a34b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.720584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.726885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.726885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.726885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.726885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.726885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.736181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5625118b-0940-490b-adba-7eb927f42ace None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:57.737520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5625118b-0940-490b-adba-7eb927f42ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:57.747725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 919/1832] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:05:57.752335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625118b-0940-490b-adba-7eb927f42ace None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.764797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625118b-0940-490b-adba-7eb927f42ace None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.785801 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.786638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.787177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.787482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.788715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:05:57.789199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.791113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.805360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 920/1833] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:05:57.809690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.816328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:57.820598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7203c11-554e-40af-9e3c-2bc6aa43061f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.825354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.826540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.826540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.826730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.827298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.829215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.832804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 914/1834] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:57.834216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.900740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c33678-efa0-4665-a2f6-f143b73afc18 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.905615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.906281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.906562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.906841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.907477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.909485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.915500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 921/1835] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:57.919834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.920357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:57.980547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 915/1836] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:05:57.984850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:57.988087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eecd4a-6b1b-466b-aa47-0e1552e45851 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.994189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:57.994800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:57.995074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:57.995254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dca0d9-fb43-41d2-a0a2-285a47280e44 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:57.995451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:57.996099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:57.998058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:57.999882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.000478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.004459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.004459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.004459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.004459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.014427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 916/1837] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:58.018574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.028811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6ff5-5339-4ac2-84f9-5ab3d77890d4 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.033455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.034148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.034428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.034703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] PATH_INFO: `/v3/projects/46ec3907f1c242eb905a9159e52486b0/users/953572ec5bd249e2adfab01802553101/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.050446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ec3907f1c242eb905a9159e52486b0, user_id=953572ec5bd249e2adfab01802553101, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.053015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.102336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 922/1838] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:05:57 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:05:58.103450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.113633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d76e4-5d81-42f8-8136-d842a9ef92eb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.123680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.124826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.125114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.130214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.130214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.130214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.142494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 917/1839] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:58 2022] PUT /identity/v3/projects/46ec3907f1c242eb905a9159e52486b0/users/953572ec5bd249e2adfab01802553101/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:05:58.147344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 923/1840] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:58.147558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.148876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.171239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6c541-eb2a-4286-8d50-0b0ff43b626b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.247855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8928-80f6-4d91-8d90-ae731ccea214 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.251686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.252438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.252726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.253000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.253650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.254539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.255159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.255266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.255591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.255877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.256544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.258861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.266570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 924/1841] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:58.270648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 918/1842] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:05:58.270784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.278296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.281359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7eb20b-cc32-4ebb-ae4e-78ecac44efc6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.288717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a30ead-d418-444c-8b6a-b419e06d1984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.289642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924f6b-a717-42bf-b61a-8a96f2a6b69a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.289796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30ead-d418-444c-8b6a-b419e06d1984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.289796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30ead-d418-444c-8b6a-b419e06d1984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.290071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30ead-d418-444c-8b6a-b419e06d1984 admin admin] PATH_INFO: `/v3/projects/1e982224ca4744afa6e2a85e4a73919c/users/c12b764892bb4f1195b026adf94cb3a1/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.295260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759112d-2bdd-432d-ad09-1e39733cd1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.296186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759112d-2bdd-432d-ad09-1e39733cd1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.296538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759112d-2bdd-432d-ad09-1e39733cd1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.296903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759112d-2bdd-432d-ad09-1e39733cd1b6 admin admin] PATH_INFO: `/v3/projects/46ec3907f1c242eb905a9159e52486b0/users/953572ec5bd249e2adfab01802553101/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.299470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759112d-2bdd-432d-ad09-1e39733cd1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ec3907f1c242eb905a9159e52486b0, user_id=953572ec5bd249e2adfab01802553101, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.302271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759112d-2bdd-432d-ad09-1e39733cd1b6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.308150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a30ead-d418-444c-8b6a-b419e06d1984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e982224ca4744afa6e2a85e4a73919c, user_id=c12b764892bb4f1195b026adf94cb3a1, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.311436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a30ead-d418-444c-8b6a-b419e06d1984 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.313795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 919/1843] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:58 2022] PUT /identity/v3/projects/46ec3907f1c242eb905a9159e52486b0/users/953572ec5bd249e2adfab01802553101/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:58.324347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.324737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.325010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.355027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] MFA Rules not processed for user `953572ec5bd249e2adfab01802553101`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:58.356261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:58.402482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 925/1844] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:05:58 2022] PUT /identity/v3/projects/1e982224ca4744afa6e2a85e4a73919c/users/c12b764892bb4f1195b026adf94cb3a1/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:05:58.412871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.413706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.414069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.433356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 920/1845] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:58.443985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] MFA Rules not processed for user `c12b764892bb4f1195b026adf94cb3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:05:58.444589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.445604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:58.446049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:58.466538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:58.474829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:05:58.504560 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff8f17-94e8-4c6b-a01a-bdfbea000cbd None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.512057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 926/1846] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:05:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:05:58.524215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.525654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:05:58.560605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.561246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.561527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.561831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.593301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be318e63-371e-4bd7-8eb7-fa4b8b2c36cb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.598094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.604891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.604891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.604891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.612660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.614138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.624578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 927/1847] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:05:58.650676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.652482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.663082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 921/1848] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:05:58.823214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:05:58.832039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3427b0-0ca7-4a0f-b7a0-1fa8cbdddb0d service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:05:58.836809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0c36c-d459-4d8b-9ed9-2a806a8f89c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:05:58.837455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c36c-d459-4d8b-9ed9-2a806a8f89c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:05:58.837761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c36c-d459-4d8b-9ed9-2a806a8f89c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:05:58.838046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c36c-d459-4d8b-9ed9-2a806a8f89c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:05:58.908163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0c36c-d459-4d8b-9ed9-2a806a8f89c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:05:58.909606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0c36c-d459-4d8b-9ed9-2a806a8f89c9 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:05:58.922268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 928/1849] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:05:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:03.064980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:03.173313 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8b568-f2fe-4e27-b7d5-d4626416d4e8 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:03.178837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:03.180216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:03.180611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:03.181255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] PATH_INFO: `/v3/users/c12b764892bb4f1195b026adf94cb3a1` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:03.182935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c12b764892bb4f1195b026adf94cb3a1)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:03.188095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:03.385403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:03.406067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c12b764892bb4f1195b026adf94cb3a1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:03.407505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c12b764892bb4f1195b026adf94cb3a1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:03.424873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c12b764892bb4f1195b026adf94cb3a1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:03.437326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c12b764892bb4f1195b026adf94cb3a1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:03.438088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c12b764892bb4f1195b026adf94cb3a1'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:03.523177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 922/1850] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:03 2022] DELETE /identity/v3/users/c12b764892bb4f1195b026adf94cb3a1 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:03.523677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:03.524134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:03.524233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:03.552126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:03.553640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:03.633184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 929/1851] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:03.640379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:03.642407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:03.733708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8079bd-d624-48cd-a349-c2b8994c802d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:03.743142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:03.743767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:03.744248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:03.744383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:03.766215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:03.767805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:03.781518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 923/1852] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:04.649977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:04.651495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:04.669135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:04.674196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8da6b7-dfa5-4670-9357-712da316bb5b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:04.690929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:04.692401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:04.693475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:04.694024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] PATH_INFO: `/v3/projects/1e982224ca4744afa6e2a85e4a73919c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:04.695627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e982224ca4744afa6e2a85e4a73919c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:04.698328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:04.854966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e982224ca4744afa6e2a85e4a73919c', 'request_id': 'req-d70b51c8-86f4-41ff-b132-fbc437877c8b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.009425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e982224ca4744afa6e2a85e4a73919c', 'request_id': 'req-d70b51c8-86f4-41ff-b132-fbc437877c8b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.017001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e982224ca4744afa6e2a85e4a73919c', 'request_id': 'req-d70b51c8-86f4-41ff-b132-fbc437877c8b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.030803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] The token cache is being invalidate because project 1e982224ca4744afa6e2a85e4a73919c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:05.031101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.032419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d70b51c8-86f4-41ff-b132-fbc437877c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.034299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 930/1853] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:04 2022] DELETE /identity/v3/projects/1e982224ca4744afa6e2a85e4a73919c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:05.091299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:05.093264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:05.107697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393844e-c73f-432a-89a0-80d9c8a6a855 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:05.192465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:05.193263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:05.193611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:05.194043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] PATH_INFO: `/v3/users/d8097c27429e4e778d47e725c9aaa083` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:05.198255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8097c27429e4e778d47e725c9aaa083)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:05.200852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:05.452869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8097c27429e4e778d47e725c9aaa083'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.453514 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8097c27429e4e778d47e725c9aaa083'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.460768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:05.471551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8097c27429e4e778d47e725c9aaa083'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.503162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8097c27429e4e778d47e725c9aaa083'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.503927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8097c27429e4e778d47e725c9aaa083'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:05.606579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 924/1854] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:05 2022] DELETE /identity/v3/users/d8097c27429e4e778d47e725c9aaa083 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:05.611269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:05.611685 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:05.613474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:05.644790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:05.650609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:05.749835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 931/1855] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:05.764801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:05.766338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:05.838624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e584bce-da36-4bcf-806f-c2bbf82e2d45 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:05.843114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:05.843834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:05.844191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:05.844530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:05.864872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:05.866950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:05.878431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 925/1856] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:06.342226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:06.346838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:06.357443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8521452-3a6f-472f-866b-78d5b135eadf None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:06.363255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:06.363932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:06.364219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:06.364485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] PATH_INFO: `/v3/projects/dc3f24b64c064b849e6dcda9b9871019` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:06.366068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3f24b64c064b849e6dcda9b9871019)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:06.368099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:06.500633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3f24b64c064b849e6dcda9b9871019', 'request_id': 'req-b2e90422-69d5-48f5-a475-be29970c0d51'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.568501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3f24b64c064b849e6dcda9b9871019', 'request_id': 'req-b2e90422-69d5-48f5-a475-be29970c0d51'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.575961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3f24b64c064b849e6dcda9b9871019', 'request_id': 'req-b2e90422-69d5-48f5-a475-be29970c0d51'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.588176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] The token cache is being invalidate because project dc3f24b64c064b849e6dcda9b9871019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:06.588505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.589186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.590734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 932/1857] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:06 2022] DELETE /identity/v3/projects/dc3f24b64c064b849e6dcda9b9871019 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:06.594890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:06.606540 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e50d5-d8cc-49e4-bfc5-32196f525e08 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:06.676409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:06.677045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:06.677324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:06.677597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] PATH_INFO: `/v3/users/953572ec5bd249e2adfab01802553101` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:06.678876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953572ec5bd249e2adfab01802553101)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:06.680788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:06.897252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953572ec5bd249e2adfab01802553101'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.898818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953572ec5bd249e2adfab01802553101'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.905018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953572ec5bd249e2adfab01802553101'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.919359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953572ec5bd249e2adfab01802553101'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.920383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0a8ca708-1ea9-4735-b6d7-a1c5df99d8d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953572ec5bd249e2adfab01802553101'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:06.923016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:07.035495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 926/1858] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:06 2022] DELETE /identity/v3/users/953572ec5bd249e2adfab01802553101 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:07.467530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:07.469149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:07.483469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e90422-69d5-48f5-a475-be29970c0d51 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:07.492252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:07.544657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:07.545546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:07.545857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:07.546134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] PATH_INFO: `/v3/projects/46ec3907f1c242eb905a9159e52486b0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:07.547256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ec3907f1c242eb905a9159e52486b0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:07.549459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:07.707331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ec3907f1c242eb905a9159e52486b0', 'request_id': 'req-b071ff6b-7e21-40f6-b9f5-53372c53f474'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:07.806428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ec3907f1c242eb905a9159e52486b0', 'request_id': 'req-b071ff6b-7e21-40f6-b9f5-53372c53f474'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:07.813329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ec3907f1c242eb905a9159e52486b0', 'request_id': 'req-b071ff6b-7e21-40f6-b9f5-53372c53f474'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:07.825231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] The token cache is being invalidate because project 46ec3907f1c242eb905a9159e52486b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:07.825541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:07.826247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b071ff6b-7e21-40f6-b9f5-53372c53f474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:07.829637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 933/1859] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:07 2022] DELETE /identity/v3/projects/46ec3907f1c242eb905a9159e52486b0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:07.835113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b654dc-8a67-4cae-b6be-cbccfd61011d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:07.836280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b654dc-8a67-4cae-b6be-cbccfd61011d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:07.836280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b654dc-8a67-4cae-b6be-cbccfd61011d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:07.865706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-94b654dc-8a67-4cae-b6be-cbccfd61011d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:07.867584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-94b654dc-8a67-4cae-b6be-cbccfd61011d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:07.930621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 927/1860] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:07.938443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818cba-1fab-4da1-b6e9-5e21404dff85 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:07.938698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818cba-1fab-4da1-b6e9-5e21404dff85 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:07.938974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818cba-1fab-4da1-b6e9-5e21404dff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:07.959518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-38818cba-1fab-4da1-b6e9-5e21404dff85 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:07.960847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-38818cba-1fab-4da1-b6e9-5e21404dff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:07.968743 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 934/1861] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:07.972953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b654dc-8a67-4cae-b6be-cbccfd61011d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:07.984036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b654dc-8a67-4cae-b6be-cbccfd61011d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:07.988639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:07.989241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:07.989522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:07.989827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:07.990630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:07.991740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:07.993340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:08.004913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 928/1862] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:08.009984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38818cba-1fab-4da1-b6e9-5e21404dff85 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:08.019329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38818cba-1fab-4da1-b6e9-5e21404dff85 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:08.024672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:08.025273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.025557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.025868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.026500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:08.028379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:08.151335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 935/1863] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:08 2022] POST /identity/v3/projects => generated 393 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:08.156082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:08.166565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820947-8515-4b4c-8909-a2012ad0b007 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:08.231095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:08.231705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.231989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.232268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.232898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:08.234865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:08.334479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 929/1864] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:08 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:08.339219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:08.349549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa3ee1-a48a-4be8-8852-ad479e2147fb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:08.354938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:08.354938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.355153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.355309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.356069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:08.357659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:08.371286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 936/1865] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:08.374947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:08.386790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8defa1-9ba4-4231-8ed8-37f4bb3c45f8 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:08.391531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227140b-da42-42c3-8106-b9f7373835f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:08.392181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227140b-da42-42c3-8106-b9f7373835f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.392469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227140b-da42-42c3-8106-b9f7373835f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.392748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227140b-da42-42c3-8106-b9f7373835f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.393399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227140b-da42-42c3-8106-b9f7373835f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:08.395096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227140b-da42-42c3-8106-b9f7373835f3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:08.406694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 930/1866] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:08.410852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:08.424898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47f59a-7c1e-4110-96f8-66cf6a349578 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:08.430664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:08.431416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.432079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.432079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] PATH_INFO: `/v3/projects/e05c811e668945f2b39de829079459e2/users/38b590c920b04577bfc1794ec89595c4/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.448238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05c811e668945f2b39de829079459e2, user_id=38b590c920b04577bfc1794ec89595c4, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:08.450930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:08.534325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 937/1867] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:08 2022] PUT /identity/v3/projects/e05c811e668945f2b39de829079459e2/users/38b590c920b04577bfc1794ec89595c4/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:08.545850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce71c-771b-466a-9f0e-3ae791321420 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.546132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce71c-771b-466a-9f0e-3ae791321420 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.546402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce71c-771b-466a-9f0e-3ae791321420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.578802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-031ce71c-771b-466a-9f0e-3ae791321420 None None] MFA Rules not processed for user `38b590c920b04577bfc1794ec89595c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:08.580881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-031ce71c-771b-466a-9f0e-3ae791321420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:08.659532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:08.679659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 931/1868] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:08.683956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:08.724893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:08.746729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19940-3b2e-4ecf-8ac6-be2ee4626161 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:08.751407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:08.752052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.752522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.753003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.754209 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:08.758047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:08.883545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 938/1869] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:08 2022] POST /identity/v3/projects => generated 391 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:08.884730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce71c-771b-466a-9f0e-3ae791321420 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:08.895611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce71c-771b-466a-9f0e-3ae791321420 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:08.963589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:08.964284 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:08.964559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:08.964831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:08.965464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:08.967448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:09.107426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 932/1870] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:08 2022] POST /identity/v3/users => generated 302 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:09.111371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:09.123818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376ccfc-5655-4200-9997-abd4cbd2358d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:09.131779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:09.132621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:09.140137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:09.140137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:09.140137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:09.140137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:09.150226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 939/1871] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:09.154832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:09.167319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1353a3-cfe7-4259-9f52-19797aa3e8e3 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:09.172403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:09.173095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:09.173374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:09.174545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] PATH_INFO: `/v3/projects/19fef600f9db4bc99c5af97b91665452/users/aa8c29c6ef304c89b4287e4bc08cefee/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:09.211692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fef600f9db4bc99c5af97b91665452, user_id=aa8c29c6ef304c89b4287e4bc08cefee, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:09.225028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:09.324385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 933/1872] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:09 2022] PUT /identity/v3/projects/19fef600f9db4bc99c5af97b91665452/users/aa8c29c6ef304c89b4287e4bc08cefee/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:09.328384 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:09.340738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eda824-448c-44f0-81d9-6de575c434e0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:09.401159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61dcd7-4945-47b5-8e7f-cd11c1e78bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:09.401800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61dcd7-4945-47b5-8e7f-cd11c1e78bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:09.402081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61dcd7-4945-47b5-8e7f-cd11c1e78bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:09.402351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61dcd7-4945-47b5-8e7f-cd11c1e78bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:09.402997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61dcd7-4945-47b5-8e7f-cd11c1e78bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:09.404569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61dcd7-4945-47b5-8e7f-cd11c1e78bd9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:09.418804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 940/1873] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:09.420303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:09.432150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce053b-8ad2-41ee-ad39-cbf540f40972 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:09.437118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:09.437823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:09.438101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:09.438372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] PATH_INFO: `/v3/projects/19fef600f9db4bc99c5af97b91665452/users/aa8c29c6ef304c89b4287e4bc08cefee/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:09.440500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fef600f9db4bc99c5af97b91665452, user_id=aa8c29c6ef304c89b4287e4bc08cefee, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:09.443250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:09.464451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 934/1874] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:09 2022] PUT /identity/v3/projects/19fef600f9db4bc99c5af97b91665452/users/aa8c29c6ef304c89b4287e4bc08cefee/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:09.466250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:09.466802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:09.467092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:09.501625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] MFA Rules not processed for user `aa8c29c6ef304c89b4287e4bc08cefee`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:09.502929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:09.588051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 941/1875] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:09 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:09.599110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:09.601413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:09.667278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35aec99-1703-44cc-bc23-949d7f3cb3f5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:09.671201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:09.671819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:09.672101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:09.672374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:09.683061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:09.684499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:09.695051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 935/1876] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:09.752079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:09.753478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:09.765773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cedf8-9989-4a28-80b6-a0847359f621 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:09.769593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac756abe-ff34-4c35-b224-c357881cef8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:09.770273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac756abe-ff34-4c35-b224-c357881cef8f service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:09.770555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac756abe-ff34-4c35-b224-c357881cef8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:09.770832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac756abe-ff34-4c35-b224-c357881cef8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:09.803199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:09.833416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac756abe-ff34-4c35-b224-c357881cef8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:09.835273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac756abe-ff34-4c35-b224-c357881cef8f service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:09.844751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 942/1877] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:16.043229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:16.091981 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:16.110456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d57b6-9623-4cf6-b0bc-a73e5e315a99 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:16.115313 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:16.116001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:16.116264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:16.116543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] PATH_INFO: `/v3/users/38b590c920b04577bfc1794ec89595c4` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:16.118208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b590c920b04577bfc1794ec89595c4)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:16.120115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:16.369624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b590c920b04577bfc1794ec89595c4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:16.370352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b590c920b04577bfc1794ec89595c4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:16.377688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b590c920b04577bfc1794ec89595c4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:16.393778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b590c920b04577bfc1794ec89595c4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:16.394465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b590c920b04577bfc1794ec89595c4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:16.477536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 936/1878] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:16 2022] DELETE /identity/v3/users/38b590c920b04577bfc1794ec89595c4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:16.483066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4362707-329e-4f03-8028-73574bc5dada None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:16.483066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4362707-329e-4f03-8028-73574bc5dada None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:16.483066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4362707-329e-4f03-8028-73574bc5dada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:16.502448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c4362707-329e-4f03-8028-73574bc5dada None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:16.506655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c4362707-329e-4f03-8028-73574bc5dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:16.578208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 943/1879] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:16.588762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:16.590213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:16.647713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79d028-8dc7-43f3-aa4a-dd52900368c3 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:16.651365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:16.651998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:16.652269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:16.652538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:16.661611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:16.663449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:16.673600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 937/1880] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:17.102613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4362707-329e-4f03-8028-73574bc5dada None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:17.104251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c4362707-329e-4f03-8028-73574bc5dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:17.123411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4362707-329e-4f03-8028-73574bc5dada None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:17.129565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:17.130529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:17.131056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:17.131626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] PATH_INFO: `/v3/projects/e05c811e668945f2b39de829079459e2` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:17.132957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05c811e668945f2b39de829079459e2)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:17.135724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:17.245974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05c811e668945f2b39de829079459e2', 'request_id': 'req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.317092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05c811e668945f2b39de829079459e2', 'request_id': 'req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.324260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05c811e668945f2b39de829079459e2', 'request_id': 'req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.335265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] The token cache is being invalidate because project e05c811e668945f2b39de829079459e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:17.335554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.336834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.338393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 944/1881] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:17 2022] DELETE /identity/v3/projects/e05c811e668945f2b39de829079459e2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:17.342394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:17.352939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa1496-d8ea-480e-bd1b-df2162eebb69 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:17.407645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:17.408268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:17.408543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:17.408813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] PATH_INFO: `/v3/users/aa8c29c6ef304c89b4287e4bc08cefee` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:17.409951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8c29c6ef304c89b4287e4bc08cefee)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:17.411848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:17.417730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:17.619871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8c29c6ef304c89b4287e4bc08cefee'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.620473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8c29c6ef304c89b4287e4bc08cefee'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.627325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8c29c6ef304c89b4287e4bc08cefee'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.640719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8c29c6ef304c89b4287e4bc08cefee'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.641427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1b08738-2f14-4fda-ba6b-ad02b55c7440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8c29c6ef304c89b4287e4bc08cefee'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:17.715886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 938/1882] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:17 2022] DELETE /identity/v3/users/aa8c29c6ef304c89b4287e4bc08cefee => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:18.355629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:18.357119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:18.371191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7f701-7c58-4e0a-9534-f755a1a1eefd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:18.443215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:18.452219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:18.453112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:18.453594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:18.454082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] PATH_INFO: `/v3/projects/19fef600f9db4bc99c5af97b91665452` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:18.455769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19fef600f9db4bc99c5af97b91665452)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:18.458588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:18.627139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19fef600f9db4bc99c5af97b91665452', 'request_id': 'req-432631c4-dffb-4569-9e9f-c60e051f3107'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:18.686754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19fef600f9db4bc99c5af97b91665452', 'request_id': 'req-432631c4-dffb-4569-9e9f-c60e051f3107'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:18.694265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19fef600f9db4bc99c5af97b91665452', 'request_id': 'req-432631c4-dffb-4569-9e9f-c60e051f3107'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:18.706612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] The token cache is being invalidate because project 19fef600f9db4bc99c5af97b91665452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:18.706909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:18.707573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432631c4-dffb-4569-9e9f-c60e051f3107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:18.709155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 945/1883] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:18 2022] DELETE /identity/v3/projects/19fef600f9db4bc99c5af97b91665452 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:18.714707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dff11-d90d-41a3-9806-38b0ee1287df None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:18.715158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dff11-d90d-41a3-9806-38b0ee1287df None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:18.715246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dff11-d90d-41a3-9806-38b0ee1287df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:18.735966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-af2dff11-d90d-41a3-9806-38b0ee1287df None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:18.737184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-af2dff11-d90d-41a3-9806-38b0ee1287df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:18.812246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 939/1884] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:18.823542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc82dc6-96b6-4559-94bb-62bc1cd21aa3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:18.823988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc82dc6-96b6-4559-94bb-62bc1cd21aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:18.824582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc82dc6-96b6-4559-94bb-62bc1cd21aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:18.848535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0cc82dc6-96b6-4559-94bb-62bc1cd21aa3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:18.849942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0cc82dc6-96b6-4559-94bb-62bc1cd21aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:18.859926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 946/1885] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:18.864345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dff11-d90d-41a3-9806-38b0ee1287df None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:18.874736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dff11-d90d-41a3-9806-38b0ee1287df None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:18.879450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:18.880046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:18.880324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:18.880597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:18.881273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:18.881688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:18.883577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:18.895407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 940/1886] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:18.900516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc82dc6-96b6-4559-94bb-62bc1cd21aa3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:18.918637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc82dc6-96b6-4559-94bb-62bc1cd21aa3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:18.918637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:18.918637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:18.918637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:18.919046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:18.919046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:18.920248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:18.960403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:18.963547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:18.976299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed75622-57ea-470f-808b-15f03ffd1398 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:18.981464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:18.982130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:18.982410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:18.982683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] PATH_INFO: `/v3/users/bde5f1c0b00248d79c77fde6f41f0b43` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:18.983910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde5f1c0b00248d79c77fde6f41f0b43)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:18.985810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:19.066828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 947/1887] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:18 2022] POST /identity/v3/projects => generated 401 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:19.070889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:19.136218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbc8b3-a2d6-4e2d-95a8-efcda8a91b8f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:19.141840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:19.142527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.142883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.143211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.144007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:19.146053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:19.239987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde5f1c0b00248d79c77fde6f41f0b43'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:19.240912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde5f1c0b00248d79c77fde6f41f0b43'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:19.247857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 948/1888] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:19 2022] POST /identity/v3/users => generated 301 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:19.253730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:19.256665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde5f1c0b00248d79c77fde6f41f0b43'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:19.262793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:19.281064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:19.288617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde5f1c0b00248d79c77fde6f41f0b43'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:19.290084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ba86b037-4dcd-46a3-8ccb-88616d97c2b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde5f1c0b00248d79c77fde6f41f0b43'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:19.293680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c5ab4-eac8-4279-be72-321cea4b5595 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:19.315669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:19.316316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.316599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.316877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.317529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:19.319360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:19.331151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 949/1889] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:19.335465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:19.347664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a23a83-e3ca-45ed-ba73-078bb25a51b5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:19.355041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:19.355846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.356391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.356391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.357038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:19.358790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:19.364286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 941/1890] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:18 2022] DELETE /identity/v3/users/bde5f1c0b00248d79c77fde6f41f0b43 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:19.369025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.369536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.370117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.371709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 950/1891] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:19.379580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:19.392563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da72b4-3edd-42b9-a7e4-062569be5ff2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:19.395571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:19.397438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:19.420829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:19.466018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 942/1892] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:19.470903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a25c8-d760-4991-9765-8c890a6160b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:19.472719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a25c8-d760-4991-9765-8c890a6160b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.472719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a25c8-d760-4991-9765-8c890a6160b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.472719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a25c8-d760-4991-9765-8c890a6160b6 admin admin] PATH_INFO: `/v3/projects/c23e2bec389d491d9dc78cfc4c04e041/users/4275ecb7099f4cb9a9262f25ae1f1f33/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.475639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:19.477244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:19.489411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a25c8-d760-4991-9765-8c890a6160b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23e2bec389d491d9dc78cfc4c04e041, user_id=4275ecb7099f4cb9a9262f25ae1f1f33, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:19.492574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a25c8-d760-4991-9765-8c890a6160b6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:19.541015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac7a5a-a2ef-435c-bf39-ceef249159f3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:19.546951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 951/1893] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:19 2022] PUT /identity/v3/projects/c23e2bec389d491d9dc78cfc4c04e041/users/4275ecb7099f4cb9a9262f25ae1f1f33/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:19.558521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12240ef2-4b2f-45ab-a275-8a6a753fd68b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.558814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12240ef2-4b2f-45ab-a275-8a6a753fd68b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.559094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12240ef2-4b2f-45ab-a275-8a6a753fd68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.594344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-12240ef2-4b2f-45ab-a275-8a6a753fd68b None None] MFA Rules not processed for user `4275ecb7099f4cb9a9262f25ae1f1f33`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:19.595735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-12240ef2-4b2f-45ab-a275-8a6a753fd68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:19.604621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:19.605257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.605540 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.605847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.665714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:19.675324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:19.676771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:19.684088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 952/1894] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:19.688089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 943/1895] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:19.690570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12240ef2-4b2f-45ab-a275-8a6a753fd68b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:19.704451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12240ef2-4b2f-45ab-a275-8a6a753fd68b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:19.710685 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:19.711771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.712185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.712467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.713119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:19.716732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:19.847725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 953/1896] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:19 2022] POST /identity/v3/projects => generated 397 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:19.849057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:19.859212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745ab8b-0116-4cce-939e-4a0bfdba5595 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:19.949646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:19.950684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:19.950974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:19.951252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:19.951901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:19.953868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.078543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 944/1897] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:19 2022] POST /identity/v3/users => generated 305 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:20.082603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:20.093529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e320-ceed-4145-b2fa-0efba6daf268 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:20.098879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:20.100023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.100023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.100509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.101851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:20.103749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.117786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 954/1898] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:20.120636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:20.131756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb580a-d885-4d6b-9d77-25bd61186da1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:20.137047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:20.138009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.140504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.140504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] PATH_INFO: `/v3/projects/e467cd5e546540e6a24c89a719540b0e/users/d04732058e3344f89eeac22ec99e72db/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.145267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:20.147111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:20.168833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58405c-de96-416f-9387-9314bbefaf11 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:20.174293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:20.175072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.175451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.175827 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] PATH_INFO: `/v3/projects/d3dd03463b6045f4b6f25f5da09df2d1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.177067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3dd03463b6045f4b6f25f5da09df2d1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:20.180574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.184161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e467cd5e546540e6a24c89a719540b0e, user_id=d04732058e3344f89eeac22ec99e72db, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:20.187008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.297442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 945/1899] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:20 2022] PUT /identity/v3/projects/e467cd5e546540e6a24c89a719540b0e/users/d04732058e3344f89eeac22ec99e72db/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:20.301408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:20.355600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3dd03463b6045f4b6f25f5da09df2d1', 'request_id': 'req-0dadeef0-1d04-41b5-9178-6633567eb5dd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.399309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265a820-ccb5-428a-a866-77d5c378cc94 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:20.437309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3dd03463b6045f4b6f25f5da09df2d1', 'request_id': 'req-0dadeef0-1d04-41b5-9178-6633567eb5dd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.439929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:20.455267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3dd03463b6045f4b6f25f5da09df2d1', 'request_id': 'req-0dadeef0-1d04-41b5-9178-6633567eb5dd'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.468757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:20.469642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.470024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.470399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.471335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:20.473784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.485176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:20.490814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 946/1900] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:20.494503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] The token cache is being invalidate because project d3dd03463b6045f4b6f25f5da09df2d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:20.495110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:20.495289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.497491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.502792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 955/1901] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:20 2022] DELETE /identity/v3/projects/d3dd03463b6045f4b6f25f5da09df2d1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:20.503609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:20.506107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df6faf-d41d-4e7d-ad77-9dda2dc7115d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:20.510419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:20.521246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246baa9-f78c-4dff-838a-6fffe42e6623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:20.522042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadeef0-1d04-41b5-9178-6633567eb5dd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:20.522597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246baa9-f78c-4dff-838a-6fffe42e6623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.522905 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246baa9-f78c-4dff-838a-6fffe42e6623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.523686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246baa9-f78c-4dff-838a-6fffe42e6623 admin admin] PATH_INFO: `/v3/projects/e467cd5e546540e6a24c89a719540b0e/users/d04732058e3344f89eeac22ec99e72db/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.526418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246baa9-f78c-4dff-838a-6fffe42e6623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e467cd5e546540e6a24c89a719540b0e, user_id=d04732058e3344f89eeac22ec99e72db, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:20.530617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:20.531446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.531871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.532293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] PATH_INFO: `/v3/users/b68a02f8890c4b0384fda1f431452664` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.533313 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246baa9-f78c-4dff-838a-6fffe42e6623 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.533958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68a02f8890c4b0384fda1f431452664)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:20.536318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.567715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 947/1902] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:20 2022] PUT /identity/v3/projects/e467cd5e546540e6a24c89a719540b0e/users/d04732058e3344f89eeac22ec99e72db/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:20.577252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.577538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.577872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.607361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] MFA Rules not processed for user `d04732058e3344f89eeac22ec99e72db`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:20.608754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:20.698011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 948/1903] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:20 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:20.707607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:20.708146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68a02f8890c4b0384fda1f431452664'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.709322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:20.710130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68a02f8890c4b0384fda1f431452664'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.718487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68a02f8890c4b0384fda1f431452664'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.730813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68a02f8890c4b0384fda1f431452664'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.731537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68a02f8890c4b0384fda1f431452664'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:20.770148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629830-af2d-45c2-8af6-bbd96a8fef5f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:20.774255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:20.775039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:20.775320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:20.775593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:20.777351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:20.792059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:20.793512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:20.804997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 949/1904] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:20.822976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 956/1905] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:20 2022] DELETE /identity/v3/users/b68a02f8890c4b0384fda1f431452664 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:21.086791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:21.088317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:21.101980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaced1-091d-4598-8fe4-f6c4f24e684d service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:21.152570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5e075-845d-421a-ad72-04219e1cf2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:21.153169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e075-845d-421a-ad72-04219e1cf2ca service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:21.153445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e075-845d-421a-ad72-04219e1cf2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:21.153717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e075-845d-421a-ad72-04219e1cf2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:21.155829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-03f5e075-845d-421a-ad72-04219e1cf2ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:21.188030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:21.218408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5e075-845d-421a-ad72-04219e1cf2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:21.219859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5e075-845d-421a-ad72-04219e1cf2ca service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:21.229437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 950/1906] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:21 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:21.356604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:21.358180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:21.467089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7ebf6-da0c-4998-8ce3-75ce4bab6ab7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:21.472473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:21.473137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:21.473421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:21.473701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] PATH_INFO: `/v3/projects/80b10694cd934436be4f0e85b265a701` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:21.474964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b10694cd934436be4f0e85b265a701)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:21.477769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:21.659707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b10694cd934436be4f0e85b265a701', 'request_id': 'req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:21.768698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b10694cd934436be4f0e85b265a701', 'request_id': 'req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:21.779048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b10694cd934436be4f0e85b265a701', 'request_id': 'req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:21.780127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:21.791635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] The token cache is being invalidate because project 80b10694cd934436be4f0e85b265a701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:21.793241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:21.793241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c1f32db5-1194-4d62-bd45-dc3ae3a7208a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:21.795165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 957/1907] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:21 2022] DELETE /identity/v3/projects/80b10694cd934436be4f0e85b265a701 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:21.799852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec0a16-dba4-4795-8bf1-07cbef88290b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:21.800792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec0a16-dba4-4795-8bf1-07cbef88290b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:21.800979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec0a16-dba4-4795-8bf1-07cbef88290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:21.839094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-29ec0a16-dba4-4795-8bf1-07cbef88290b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:21.840443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-29ec0a16-dba4-4795-8bf1-07cbef88290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:21.911351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 951/1908] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:21.920773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4e1ce-f4e5-4aab-9775-f1a9223e3287 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:21.921163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4e1ce-f4e5-4aab-9775-f1a9223e3287 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:21.921534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4e1ce-f4e5-4aab-9775-f1a9223e3287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:21.948900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f6a4e1ce-f4e5-4aab-9775-f1a9223e3287 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:21.950557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f6a4e1ce-f4e5-4aab-9775-f1a9223e3287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:21.958468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 958/1909] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:21.963188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec0a16-dba4-4795-8bf1-07cbef88290b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:21.973214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec0a16-dba4-4795-8bf1-07cbef88290b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:21.978251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:21.978859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:21.979141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:21.979416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:21.980067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:21.980370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:21.981992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:21.994484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 952/1910] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:06:21.999014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4e1ce-f4e5-4aab-9775-f1a9223e3287 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.009047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4e1ce-f4e5-4aab-9775-f1a9223e3287 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.013878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.014383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.014658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.014927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.015598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.017505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.114015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 959/1911] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:21 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:22.119278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.129372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7054d00-5369-4844-98da-85435d060d4a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.187095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238a9d-56ab-4001-82a2-646ca011d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.187799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238a9d-56ab-4001-82a2-646ca011d6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.188024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238a9d-56ab-4001-82a2-646ca011d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.188301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238a9d-56ab-4001-82a2-646ca011d6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.188927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238a9d-56ab-4001-82a2-646ca011d6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.190916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238a9d-56ab-4001-82a2-646ca011d6f2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.269495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 953/1912] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:22 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:22.274137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.284321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff89fc-5122-4523-93fb-3728d769d83e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.289829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d419-834b-40a6-884a-70511cbc4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.290266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea541f-e37c-4c56-89b2-29d025d91d4e None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.290559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea541f-e37c-4c56-89b2-29d025d91d4e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.290647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d419-834b-40a6-884a-70511cbc4968 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.290890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea541f-e37c-4c56-89b2-29d025d91d4e None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.291001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d419-834b-40a6-884a-70511cbc4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.291199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d419-834b-40a6-884a-70511cbc4968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.291852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d419-834b-40a6-884a-70511cbc4968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.292495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 954/1913] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:06:22.293484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d419-834b-40a6-884a-70511cbc4968 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.303980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 960/1914] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:22.308401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea541f-e37c-4c56-89b2-29d025d91d4e None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.321619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea541f-e37c-4c56-89b2-29d025d91d4e None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.324663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.325042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.325329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.327222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.330217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.330217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.330217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.330217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.331176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.343758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 955/1915] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:22.347957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.354450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:22.355748 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:22.360361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f35ffc-ae11-4c0a-9cce-b99802ce26e5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.365510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.366205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.366482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.367516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] PATH_INFO: `/v3/projects/179a5cf4f5564dcea420628fd37e9b7e/users/98aba1bd305a40208fb0ab724b9dcf2c/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.383197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179a5cf4f5564dcea420628fd37e9b7e, user_id=98aba1bd305a40208fb0ab724b9dcf2c, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.385886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.421228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 961/1916] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:06:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:22.436099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.437625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:22.451031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003893f-da90-4c8c-a62e-a3ce8d007264 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.457546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0d6d-0609-4fad-bfe6-7144fdfcbcf4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.462340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0d6d-0609-4fad-bfe6-7144fdfcbcf4 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.462340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0d6d-0609-4fad-bfe6-7144fdfcbcf4 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.462340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0d6d-0609-4fad-bfe6-7144fdfcbcf4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.464896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 956/1917] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:22 2022] PUT /identity/v3/projects/179a5cf4f5564dcea420628fd37e9b7e/users/98aba1bd305a40208fb0ab724b9dcf2c/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:22.466008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.467723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:22.467942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0d6d-0609-4fad-bfe6-7144fdfcbcf4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.469462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0d6d-0609-4fad-bfe6-7144fdfcbcf4 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.481269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 962/1918] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:22.482904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67515b4-83c3-4588-abf8-a48effe0fd9c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.483196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67515b4-83c3-4588-abf8-a48effe0fd9c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.483487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67515b4-83c3-4588-abf8-a48effe0fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.494696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:22.519787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c67515b4-83c3-4588-abf8-a48effe0fd9c None None] MFA Rules not processed for user `98aba1bd305a40208fb0ab724b9dcf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:22.521018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c67515b4-83c3-4588-abf8-a48effe0fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:22.550868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315030b-284d-4d97-bb99-602c0a7d5c79 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.555641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.556271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.556556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.556826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] PATH_INFO: `/v3/users/e54f17d6995f4c099708d79715d95220` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.557983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e54f17d6995f4c099708d79715d95220)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.559920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.589612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 963/1919] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:22.594261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67515b4-83c3-4588-abf8-a48effe0fd9c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.604695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67515b4-83c3-4588-abf8-a48effe0fd9c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.609294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.609930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.610217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.610492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.611119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.613013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.713150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 964/1920] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:22 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:22.717189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.748466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e54f17d6995f4c099708d79715d95220'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:22.749138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e54f17d6995f4c099708d79715d95220'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:22.756111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e54f17d6995f4c099708d79715d95220'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:22.768381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e54f17d6995f4c099708d79715d95220'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:22.769122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e54f17d6995f4c099708d79715d95220'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:22.780574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49607e-9587-481f-9c38-f783753a18b3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:22.785327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:22.785992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:22.786277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:22.786543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:22.787200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:22.789166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:22.894475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 957/1921] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:22 2022] DELETE /identity/v3/users/e54f17d6995f4c099708d79715d95220 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:22.936463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 965/1922] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:22 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:22.937730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:22.939169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:22.989564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:23.024942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17faa551-5577-428e-9cf8-ef8a0d074907 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.029869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.030559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.030835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.031106 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.031746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.033341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.044104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 958/1923] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:23.048582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.059657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64547e7d-1336-4324-8bf8-e799e1b42226 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.064294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.064952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.067974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.067974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] PATH_INFO: `/v3/projects/0ae0a809de8141fbafa9f3881447361d/users/7d6070a5adc24e258082eecde53a8b2a/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.080229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae0a809de8141fbafa9f3881447361d, user_id=7d6070a5adc24e258082eecde53a8b2a, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.083001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.174037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 966/1924] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:23 2022] PUT /identity/v3/projects/0ae0a809de8141fbafa9f3881447361d/users/7d6070a5adc24e258082eecde53a8b2a/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:23.178351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.178499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.181282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.198571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1ebd3-5995-48b9-aeb0-c6c1c8bef8c1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.232066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:23.272215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baff9e3-74be-4420-b984-4ae63c09e6b0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.273238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.275013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.275464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.277894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] PATH_INFO: `/v3/projects/eaaf8aa87a0f47cc9ba9f477de3c2d67` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.277894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaaf8aa87a0f47cc9ba9f477de3c2d67)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.278267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.278369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.278821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.279106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.280184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.280477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.283009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.302087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 967/1925] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:23.308273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.319722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac975e-ed35-47f1-85da-b2eb2e7ba2f6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.326484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8ab8d-5049-4ce7-87d6-8b683ddb897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.327217 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8ab8d-5049-4ce7-87d6-8b683ddb897a admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.327570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8ab8d-5049-4ce7-87d6-8b683ddb897a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.327911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8ab8d-5049-4ce7-87d6-8b683ddb897a admin admin] PATH_INFO: `/v3/projects/0ae0a809de8141fbafa9f3881447361d/users/7d6070a5adc24e258082eecde53a8b2a/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.330112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8ab8d-5049-4ce7-87d6-8b683ddb897a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae0a809de8141fbafa9f3881447361d, user_id=7d6070a5adc24e258082eecde53a8b2a, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.332802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8ab8d-5049-4ce7-87d6-8b683ddb897a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.346928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 968/1926] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:23 2022] PUT /identity/v3/projects/0ae0a809de8141fbafa9f3881447361d/users/7d6070a5adc24e258082eecde53a8b2a/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:23.358924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.359212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.359495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.387615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] MFA Rules not processed for user `7d6070a5adc24e258082eecde53a8b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:23.388933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.403873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaaf8aa87a0f47cc9ba9f477de3c2d67', 'request_id': 'req-7930a678-308f-41e8-ac30-4e395416eed3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:23.434190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:23.464089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 969/1927] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:23.475295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.482330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.503231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaaf8aa87a0f47cc9ba9f477de3c2d67', 'request_id': 'req-7930a678-308f-41e8-ac30-4e395416eed3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:23.510927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaaf8aa87a0f47cc9ba9f477de3c2d67', 'request_id': 'req-7930a678-308f-41e8-ac30-4e395416eed3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:23.521634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] The token cache is being invalidate because project eaaf8aa87a0f47cc9ba9f477de3c2d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:23.521958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:23.522615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7930a678-308f-41e8-ac30-4e395416eed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:23.524649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 959/1928] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:23 2022] DELETE /identity/v3/projects/eaaf8aa87a0f47cc9ba9f477de3c2d67 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:23.529517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daaebf-18c2-496a-bda7-5a620180a6a0 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.529813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daaebf-18c2-496a-bda7-5a620180a6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.530096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daaebf-18c2-496a-bda7-5a620180a6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.535552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc40a3-787c-4d68-971f-a9b3312105ca None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.539927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86d0d9-5169-49a6-a990-d6685e1879fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.540521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86d0d9-5169-49a6-a990-d6685e1879fb service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.540783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86d0d9-5169-49a6-a990-d6685e1879fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.541059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86d0d9-5169-49a6-a990-d6685e1879fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.542836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ad86d0d9-5169-49a6-a990-d6685e1879fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.550668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-16daaebf-18c2-496a-bda7-5a620180a6a0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:23.551835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-16daaebf-18c2-496a-bda7-5a620180a6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.555036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86d0d9-5169-49a6-a990-d6685e1879fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.556462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86d0d9-5169-49a6-a990-d6685e1879fb service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.566262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 970/1929] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:23.613608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 960/1930] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:23.621621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab601-a847-473f-b23e-9252fc91226b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.621977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab601-a847-473f-b23e-9252fc91226b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.622283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab601-a847-473f-b23e-9252fc91226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.642254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e4eab601-a847-473f-b23e-9252fc91226b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:23.643460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e4eab601-a847-473f-b23e-9252fc91226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.652181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 971/1931] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:23.656490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daaebf-18c2-496a-bda7-5a620180a6a0 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.666927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daaebf-18c2-496a-bda7-5a620180a6a0 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.672853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.673525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.673846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.674134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.674840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:23.675153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.676762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.689215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 961/1932] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:06:23.693908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eab601-a847-473f-b23e-9252fc91226b None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.703754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eab601-a847-473f-b23e-9252fc91226b None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.708450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.709048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.709325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.709596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.710270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.712726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.792975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.795023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.808956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 972/1933] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:23 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:23.810274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb62ed-3701-449e-b83d-3c1e1f63a411 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.814356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:23.860798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:23.869009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.869605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.869926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.870205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.871965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:23.895659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a466b3b-9d63-4a67-85be-11d7dd72e34f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:23.902762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:23.902928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:23.903267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:23.903631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:23.904429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.906426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.938311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:23.939774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:23.949631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 962/1934] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:24.043269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 973/1935] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:23 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:24.047839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:24.058864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dae19-25c4-48f3-8684-e54e506adfd0 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:24.063434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641500c-c648-427e-af38-07735bff722e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:24.064031 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641500c-c648-427e-af38-07735bff722e admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:24.064310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641500c-c648-427e-af38-07735bff722e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:24.064612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641500c-c648-427e-af38-07735bff722e admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:24.065249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641500c-c648-427e-af38-07735bff722e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:24.066858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641500c-c648-427e-af38-07735bff722e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:24.079258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 963/1936] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:24.083288 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:24.094544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02901979-c472-47b2-be11-2c3dd1c38e29 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:24.099483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d6c37-58ba-4153-870a-5f97e3121170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:24.100168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d6c37-58ba-4153-870a-5f97e3121170 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:24.100514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d6c37-58ba-4153-870a-5f97e3121170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:24.100854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d6c37-58ba-4153-870a-5f97e3121170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:24.101571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d6c37-58ba-4153-870a-5f97e3121170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:24.103290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d6c37-58ba-4153-870a-5f97e3121170 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:24.117560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 974/1937] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:24.118930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641500c-c648-427e-af38-07735bff722e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:24.129928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641500c-c648-427e-af38-07735bff722e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:24.135606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:24.136264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:24.136543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:24.136818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] PATH_INFO: `/v3/projects/942aba47b5e2439a9bdd2e52f25f3290/users/b496850076a5466a9f7e07aee3bc1ed4/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:24.152744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942aba47b5e2439a9bdd2e52f25f3290, user_id=b496850076a5466a9f7e07aee3bc1ed4, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:24.155505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:24.275271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 964/1938] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:24 2022] PUT /identity/v3/projects/942aba47b5e2439a9bdd2e52f25f3290/users/b496850076a5466a9f7e07aee3bc1ed4/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:24.285476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ccdf3-2e9a-4df3-ad0e-b210f6cc637d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:24.286070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ccdf3-2e9a-4df3-ad0e-b210f6cc637d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:24.286353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ccdf3-2e9a-4df3-ad0e-b210f6cc637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:24.316612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-253ccdf3-2e9a-4df3-ad0e-b210f6cc637d None None] MFA Rules not processed for user `b496850076a5466a9f7e07aee3bc1ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:24.317851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-253ccdf3-2e9a-4df3-ad0e-b210f6cc637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:24.331192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:24.384408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 975/1939] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:24.394491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:24.396184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:24.477379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c13f-8c70-4737-8c0f-c1fc1cb9def3 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:24.482872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857db35-eb30-4fa4-81d8-2920a446001e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:24.483504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857db35-eb30-4fa4-81d8-2920a446001e service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:24.483779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857db35-eb30-4fa4-81d8-2920a446001e service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:24.484047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857db35-eb30-4fa4-81d8-2920a446001e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:24.492707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857db35-eb30-4fa4-81d8-2920a446001e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:24.494306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857db35-eb30-4fa4-81d8-2920a446001e service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:24.504577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 965/1940] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:24.726847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ccdf3-2e9a-4df3-ad0e-b210f6cc637d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:24.789596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ccdf3-2e9a-4df3-ad0e-b210f6cc637d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:24.794171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:24.795019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:24.795300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:24.795577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] PATH_INFO: `/v3/users/b496850076a5466a9f7e07aee3bc1ed4` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:24.796685 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b496850076a5466a9f7e07aee3bc1ed4)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:24.798617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:25.068664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b496850076a5466a9f7e07aee3bc1ed4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:25.069229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b496850076a5466a9f7e07aee3bc1ed4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:25.076914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b496850076a5466a9f7e07aee3bc1ed4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:25.090182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b496850076a5466a9f7e07aee3bc1ed4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:25.090895 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b496850076a5466a9f7e07aee3bc1ed4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:25.162470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 976/1941] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:24 2022] DELETE /identity/v3/users/b496850076a5466a9f7e07aee3bc1ed4 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:25.167722 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:25.168184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:25.168459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:25.187581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:25.188794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:25.234767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:25.249427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 966/1942] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:25.259859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:25.261514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:25.320891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:25.340649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f9f06-13cb-4db3-9b48-48897c1d2892 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:25.344554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a3046-03b8-48ea-a08e-95b1d89b2085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:25.345162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3046-03b8-48ea-a08e-95b1d89b2085 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:25.345565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3046-03b8-48ea-a08e-95b1d89b2085 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:25.345879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3046-03b8-48ea-a08e-95b1d89b2085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:25.355508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a3046-03b8-48ea-a08e-95b1d89b2085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:25.357051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a3046-03b8-48ea-a08e-95b1d89b2085 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:25.366828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 977/1943] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:25.777281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:25.778821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:25.791644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca3f67-2144-497f-bda5-f0da0a64a143 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:25.798250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:25.798895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:25.799177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:25.799565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] PATH_INFO: `/v3/projects/942aba47b5e2439a9bdd2e52f25f3290` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:25.800652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942aba47b5e2439a9bdd2e52f25f3290)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:25.802639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:25.993313 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942aba47b5e2439a9bdd2e52f25f3290', 'request_id': 'req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.086008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942aba47b5e2439a9bdd2e52f25f3290', 'request_id': 'req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.092887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942aba47b5e2439a9bdd2e52f25f3290', 'request_id': 'req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.104101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] The token cache is being invalidate because project 942aba47b5e2439a9bdd2e52f25f3290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:26.104388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.105105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c5ec6a64-fe8f-480a-8867-6b0f10ffbe51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.106643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 967/1944] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:25 2022] DELETE /identity/v3/projects/942aba47b5e2439a9bdd2e52f25f3290 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:26.112156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56aa2c-50de-40a9-9fbb-80407194837d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.112597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56aa2c-50de-40a9-9fbb-80407194837d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.112823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56aa2c-50de-40a9-9fbb-80407194837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.133910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fc56aa2c-50de-40a9-9fbb-80407194837d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:26.135026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc56aa2c-50de-40a9-9fbb-80407194837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:26.196099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 978/1945] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:26.205166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07bd0-559f-40d3-b87f-cde5eb24cc70 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.205454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07bd0-559f-40d3-b87f-cde5eb24cc70 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.205754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c07bd0-559f-40d3-b87f-cde5eb24cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.225050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-45c07bd0-559f-40d3-b87f-cde5eb24cc70 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:26.226986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-45c07bd0-559f-40d3-b87f-cde5eb24cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:26.234807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 968/1946] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:26.239095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56aa2c-50de-40a9-9fbb-80407194837d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.249239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56aa2c-50de-40a9-9fbb-80407194837d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.253881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.254483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.254761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.255032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.255670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:26.255971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.257607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:26.269652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 979/1947] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:26.273835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c07bd0-559f-40d3-b87f-cde5eb24cc70 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.282873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c07bd0-559f-40d3-b87f-cde5eb24cc70 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.287441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.288043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.288317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.289137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.289802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.291681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:26.414157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 969/1948] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:26 2022] POST /identity/v3/projects => generated 405 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:26.418957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.429697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0651009-73f5-4e01-b7ed-c9e80723c78a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.503041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.503041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.503041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.503041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.503372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.503443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:26.592232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 980/1949] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:26 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:26.595655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.606014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79b665-8494-4aa9-bee9-b2c47deff108 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.610830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.611432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.611712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.611986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.612667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.614303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:26.624669 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 970/1950] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:26.628617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.639542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31406000-397a-4a79-a3d9-b8b60b451a35 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.644561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.645238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.645531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.645829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.646523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.648482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:26.659576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 981/1951] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:26.663903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.674386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc280-93b3-45a3-ac22-12b1236b410a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.680195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f763036-e2c5-4991-abd3-7007a5533360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.680937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f763036-e2c5-4991-abd3-7007a5533360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.681214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f763036-e2c5-4991-abd3-7007a5533360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.681517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f763036-e2c5-4991-abd3-7007a5533360 admin admin] PATH_INFO: `/v3/projects/de5242a56ff14149ba6f6195c7e03339/users/6969bb9c2ea9462d93d5b4b55de54db7/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.685379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.687082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:26.697782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f763036-e2c5-4991-abd3-7007a5533360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5242a56ff14149ba6f6195c7e03339, user_id=6969bb9c2ea9462d93d5b4b55de54db7, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.700376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d898064-821e-45d7-8c32-516b28302e78 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.700632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f763036-e2c5-4991-abd3-7007a5533360 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:26.704960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.705570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.705879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.706152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] PATH_INFO: `/v3/users/4275ecb7099f4cb9a9262f25ae1f1f33` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.707268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4275ecb7099f4cb9a9262f25ae1f1f33)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.709142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:26.803178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 971/1952] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:26 2022] PUT /identity/v3/projects/de5242a56ff14149ba6f6195c7e03339/users/6969bb9c2ea9462d93d5b4b55de54db7/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:26.816945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fd3d-1b68-4a24-b1cb-c9d4a9ea3f2a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.817230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fd3d-1b68-4a24-b1cb-c9d4a9ea3f2a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.817535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42fd3d-1b68-4a24-b1cb-c9d4a9ea3f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.846610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-cd42fd3d-1b68-4a24-b1cb-c9d4a9ea3f2a None None] MFA Rules not processed for user `6969bb9c2ea9462d93d5b4b55de54db7`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:26.847823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cd42fd3d-1b68-4a24-b1cb-c9d4a9ea3f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:26.906080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:26.911469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 972/1953] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:26.923150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fd3d-1b68-4a24-b1cb-c9d4a9ea3f2a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:26.923307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:26.927089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4275ecb7099f4cb9a9262f25ae1f1f33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.927781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4275ecb7099f4cb9a9262f25ae1f1f33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.935271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4275ecb7099f4cb9a9262f25ae1f1f33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.948610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4275ecb7099f4cb9a9262f25ae1f1f33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.949315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7e261b93-4900-40f1-a684-7dba69a3ad39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4275ecb7099f4cb9a9262f25ae1f1f33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:26.980525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42fd3d-1b68-4a24-b1cb-c9d4a9ea3f2a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:26.985107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:26.985712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:26.986021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:26.986294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:26.986925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:26.988879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.023019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 982/1954] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:26 2022] DELETE /identity/v3/users/4275ecb7099f4cb9a9262f25ae1f1f33 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:27.027468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.027783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.028063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.046957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:27.048140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:27.105812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 973/1955] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:26 2022] POST /identity/v3/projects => generated 405 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:27.107818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 983/1956] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:27.112814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.114416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:27.118576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.120138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:27.179305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd47bd-4869-4ef2-bf8c-f86d70978ff0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.179884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3b4fa-36a7-484b-b114-465247d44cb1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.184575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.185183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.185468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.185757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.186920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.188471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.229453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.230098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.230378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.230652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.239639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.241067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.250465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 984/1957] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:27.263807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 974/1958] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:27 2022] POST /identity/v3/users => generated 309 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:27.267585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.277637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121cc68-0412-473c-aed6-38f12decdb67 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.283889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.284647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.284976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.285333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.286147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.288081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.300012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 985/1959] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:27.304090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.314420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffb43f-1268-46da-95c6-04c3db6e312c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.319240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.319848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.320127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.320439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] PATH_INFO: `/v3/projects/a36c55a6288549e6ac718f3d5370d4b8/users/acb608285a464b1db9816369d6e76f92/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.335950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36c55a6288549e6ac718f3d5370d4b8, user_id=acb608285a464b1db9816369d6e76f92, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.338906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.408872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 975/1960] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:27 2022] PUT /identity/v3/projects/a36c55a6288549e6ac718f3d5370d4b8/users/acb608285a464b1db9816369d6e76f92/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:27.413030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.420864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:27.424717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170729f-f1c1-4ef8-bf0c-97be79b883fd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.487038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1144972-aa2a-4886-8ecb-9bf4931a1548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.487760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144972-aa2a-4886-8ecb-9bf4931a1548 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.488038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144972-aa2a-4886-8ecb-9bf4931a1548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.488309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1144972-aa2a-4886-8ecb-9bf4931a1548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.489073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1144972-aa2a-4886-8ecb-9bf4931a1548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.490937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1144972-aa2a-4886-8ecb-9bf4931a1548 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.501465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 986/1961] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:27.505770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.516671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa9119-994b-4f03-94e4-06f5f4ea3a99 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.522378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.523801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.524176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.524633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] PATH_INFO: `/v3/projects/a36c55a6288549e6ac718f3d5370d4b8/users/acb608285a464b1db9816369d6e76f92/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.527408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36c55a6288549e6ac718f3d5370d4b8, user_id=acb608285a464b1db9816369d6e76f92, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.529913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.553388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 976/1962] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:27 2022] PUT /identity/v3/projects/a36c55a6288549e6ac718f3d5370d4b8/users/acb608285a464b1db9816369d6e76f92/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:27.555939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.556219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.556604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.596090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] MFA Rules not processed for user `acb608285a464b1db9816369d6e76f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:27.597350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:27.690130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 987/1963] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:27.691468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.693942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:27.700383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.701859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:27.770618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:27.786977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169009fb-dce5-44c1-a8e5-2574a8df5a42 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.791665 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.792303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.805456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.805456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.805658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda04af6-c5e8-4b7a-9094-d252225192a6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.805658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.806223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.806329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.806329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.806329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] PATH_INFO: `/v3/projects/c23e2bec389d491d9dc78cfc4c04e041` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.806548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.806629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23e2bec389d491d9dc78cfc4c04e041)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.808542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.814267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 988/1964] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:27 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:27.847693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:27.849140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:27.863772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981775b-dc47-4195-a586-8222e929bffa service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:27.868614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:27.869255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:27.869531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:27.869834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] PATH_INFO: `/v3/users/98aba1bd305a40208fb0ab724b9dcf2c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:27.870978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=98aba1bd305a40208fb0ab724b9dcf2c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:27.872875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:27.896216 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:28.052065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98aba1bd305a40208fb0ab724b9dcf2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.052463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23e2bec389d491d9dc78cfc4c04e041', 'request_id': 'req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.052727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98aba1bd305a40208fb0ab724b9dcf2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.059995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98aba1bd305a40208fb0ab724b9dcf2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.072221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98aba1bd305a40208fb0ab724b9dcf2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.072906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98aba1bd305a40208fb0ab724b9dcf2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.122889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23e2bec389d491d9dc78cfc4c04e041', 'request_id': 'req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.129803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23e2bec389d491d9dc78cfc4c04e041', 'request_id': 'req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.140301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] The token cache is being invalidate because project c23e2bec389d491d9dc78cfc4c04e041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:28.140583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.141237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.142738 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 977/1965] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:27 2022] DELETE /identity/v3/projects/c23e2bec389d491d9dc78cfc4c04e041 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:28.145691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:28.145691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:28.185865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 989/1966] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:27 2022] DELETE /identity/v3/users/98aba1bd305a40208fb0ab724b9dcf2c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:28.186865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:28.188397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:28.216004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c63bc-73b4-4a65-ae7c-ced752e2ad3a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:28.252255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbdcd1-00eb-4474-b592-7ec11cfecdbd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:28.259208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:28.259836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:28.260120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:28.260451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] PATH_INFO: `/v3/users/d04732058e3344f89eeac22ec99e72db` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:28.261569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04732058e3344f89eeac22ec99e72db)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:28.263505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:28.274926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:28.275568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:28.275849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:28.276129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] PATH_INFO: `/v3/users/6969bb9c2ea9462d93d5b4b55de54db7` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:28.277248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6969bb9c2ea9462d93d5b4b55de54db7)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:28.279192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:28.292904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:28.507535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04732058e3344f89eeac22ec99e72db'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.508215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04732058e3344f89eeac22ec99e72db'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.515358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04732058e3344f89eeac22ec99e72db'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.527504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04732058e3344f89eeac22ec99e72db'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.528171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-369da80a-a642-40e8-9898-d15143d1806e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04732058e3344f89eeac22ec99e72db'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.660299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 990/1967] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:28 2022] DELETE /identity/v3/users/d04732058e3344f89eeac22ec99e72db => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:28.661569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:28.661892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:28.662222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:28.670977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6969bb9c2ea9462d93d5b4b55de54db7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.672707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6969bb9c2ea9462d93d5b4b55de54db7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.681336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6969bb9c2ea9462d93d5b4b55de54db7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.682956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:28.684170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:28.697072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6969bb9c2ea9462d93d5b4b55de54db7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.697876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-12460013-1da8-4ec8-830e-d18f0f04b89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6969bb9c2ea9462d93d5b4b55de54db7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:28.700004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:28.746874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 991/1968] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:28.756509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:28.758213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:28.809034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 978/1969] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:28 2022] DELETE /identity/v3/users/6969bb9c2ea9462d93d5b4b55de54db7 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:28.813698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1a4e-65df-4559-8442-98e05f469ad4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:28.814017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1a4e-65df-4559-8442-98e05f469ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:28.814288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1a4e-65df-4559-8442-98e05f469ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:28.821458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01fbb-7098-46df-973f-39e110f7ceb0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:28.834581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-acda1a4e-65df-4559-8442-98e05f469ad4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:28.843473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-acda1a4e-65df-4559-8442-98e05f469ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:28.883711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d14d2-6708-4981-96c0-bbbfb19a8fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:28.884704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d14d2-6708-4981-96c0-bbbfb19a8fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:28.885134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d14d2-6708-4981-96c0-bbbfb19a8fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:28.889873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d14d2-6708-4981-96c0-bbbfb19a8fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:28.954116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 979/1970] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:28.964906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1a4e-65df-4559-8442-98e05f469ad4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:28.976685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1a4e-65df-4559-8442-98e05f469ad4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:28.980408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:28.981361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:28.981361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:28.981576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:28.991190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:28.995829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:29.002521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d14d2-6708-4981-96c0-bbbfb19a8fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:29.003661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 980/1971] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:29.003849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d14d2-6708-4981-96c0-bbbfb19a8fa2 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:29.015295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 992/1972] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:29.158634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:29.160172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:29.175360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e362a8d-8feb-4f4d-b386-102beeee4c4d service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:29.181465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:29.182295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:29.182576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:29.182847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] PATH_INFO: `/v3/projects/e467cd5e546540e6a24c89a719540b0e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:29.184035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e467cd5e546540e6a24c89a719540b0e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:29.187282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:29.362824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e467cd5e546540e6a24c89a719540b0e', 'request_id': 'req-b49268e7-7131-421b-b426-e145545d01bc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.440834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e467cd5e546540e6a24c89a719540b0e', 'request_id': 'req-b49268e7-7131-421b-b426-e145545d01bc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.450030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e467cd5e546540e6a24c89a719540b0e', 'request_id': 'req-b49268e7-7131-421b-b426-e145545d01bc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.462674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] The token cache is being invalidate because project e467cd5e546540e6a24c89a719540b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:29.462962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.463628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b49268e7-7131-421b-b426-e145545d01bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.465201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 981/1973] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:29 2022] DELETE /identity/v3/projects/e467cd5e546540e6a24c89a719540b0e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:29.470246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67892ea-4319-4500-9a50-378acb39b773 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:29.470719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67892ea-4319-4500-9a50-378acb39b773 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:29.470914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67892ea-4319-4500-9a50-378acb39b773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:29.492556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-c67892ea-4319-4500-9a50-378acb39b773 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:29.493884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c67892ea-4319-4500-9a50-378acb39b773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:29.518403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:29.554230 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 993/1974] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:29.562773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabae90-0e2b-4f14-a924-5a72dfdd8a4a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:29.563113 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabae90-0e2b-4f14-a924-5a72dfdd8a4a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:29.563391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabae90-0e2b-4f14-a924-5a72dfdd8a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:29.572624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:29.586858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7dabae90-0e2b-4f14-a924-5a72dfdd8a4a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:29.588086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7dabae90-0e2b-4f14-a924-5a72dfdd8a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:29.596201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 982/1975] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:29.600219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67892ea-4319-4500-9a50-378acb39b773 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:29.611365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67892ea-4319-4500-9a50-378acb39b773 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:29.616147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:29.617907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:29.623583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:29.623583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:29.623583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:29.623583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:29.623583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:29.633036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 994/1976] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:29.639686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabae90-0e2b-4f14-a924-5a72dfdd8a4a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:29.649166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabae90-0e2b-4f14-a924-5a72dfdd8a4a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:29.653824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:29.656955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:29.656955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:29.656955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:29.656955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:29.657424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:29.713521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:29.715397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:29.729175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2d4fb-99b0-43c8-ac59-306958073a41 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:29.734737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:29.735458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:29.735739 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:29.736014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] PATH_INFO: `/v3/projects/179a5cf4f5564dcea420628fd37e9b7e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:29.737341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=179a5cf4f5564dcea420628fd37e9b7e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:29.739243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:29.770584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 983/1977] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:29 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:29.771431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:29.773040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:29.836727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41fbde-656a-47ad-bb2d-ee64f97f8a57 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:29.841382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:29.842056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:29.842337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:29.842609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] PATH_INFO: `/v3/projects/de5242a56ff14149ba6f6195c7e03339` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:29.843722 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5242a56ff14149ba6f6195c7e03339)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:29.845608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:29.867137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179a5cf4f5564dcea420628fd37e9b7e', 'request_id': 'req-e1573240-09e1-4841-80ea-68c76b682d69'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.932059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179a5cf4f5564dcea420628fd37e9b7e', 'request_id': 'req-e1573240-09e1-4841-80ea-68c76b682d69'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.939384 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179a5cf4f5564dcea420628fd37e9b7e', 'request_id': 'req-e1573240-09e1-4841-80ea-68c76b682d69'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.950656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] The token cache is being invalidate because project 179a5cf4f5564dcea420628fd37e9b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:29.950956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.951603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:29.953125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 995/1978] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:29 2022] DELETE /identity/v3/projects/179a5cf4f5564dcea420628fd37e9b7e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:29.953902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:29.955163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:29.980606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5242a56ff14149ba6f6195c7e03339', 'request_id': 'req-f81a70ec-b806-4049-9828-129e127bcb85'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.027898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573240-09e1-4841-80ea-68c76b682d69 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:30.057341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5242a56ff14149ba6f6195c7e03339', 'request_id': 'req-f81a70ec-b806-4049-9828-129e127bcb85'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.065184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5242a56ff14149ba6f6195c7e03339', 'request_id': 'req-f81a70ec-b806-4049-9828-129e127bcb85'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.074894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:30.077680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] The token cache is being invalidate because project de5242a56ff14149ba6f6195c7e03339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:30.078640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.078640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.080572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 984/1979] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:29 2022] DELETE /identity/v3/projects/de5242a56ff14149ba6f6195c7e03339 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:30.083973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:30.087056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:30.092062 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:30.092906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:30.093369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:30.093847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:30.094655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:30.096840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:30.099781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a70ec-b806-4049-9828-129e127bcb85 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:30.105341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:30.106023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:30.106302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:30.106572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] PATH_INFO: `/v3/users/7d6070a5adc24e258082eecde53a8b2a` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:30.107706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6070a5adc24e258082eecde53a8b2a)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:30.109604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:30.220352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 996/1980] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:29 2022] POST /identity/v3/users => generated 291 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:30.221256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:30.222855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:30.234486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ba2e0-b8d3-4dc5-a196-1ce844b04d16 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:30.239430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:30.240056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:30.240338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:30.240616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] PATH_INFO: `/v3/users/acb608285a464b1db9816369d6e76f92` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:30.242452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb608285a464b1db9816369d6e76f92)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:30.244440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:30.362473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6070a5adc24e258082eecde53a8b2a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.363138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6070a5adc24e258082eecde53a8b2a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.370851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6070a5adc24e258082eecde53a8b2a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.383003 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6070a5adc24e258082eecde53a8b2a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.383663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6070a5adc24e258082eecde53a8b2a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.456779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb608285a464b1db9816369d6e76f92'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.457470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb608285a464b1db9816369d6e76f92'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.464624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb608285a464b1db9816369d6e76f92'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.476767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb608285a464b1db9816369d6e76f92'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.477420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb608285a464b1db9816369d6e76f92'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:30.486159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 985/1981] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:30 2022] DELETE /identity/v3/users/7d6070a5adc24e258082eecde53a8b2a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:30.487249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:30.488647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:30.555608 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4878d0-0a13-479d-a1e4-ff59c40fe460 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:30.561535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:30.562665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:30.562951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:30.563253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:30.564051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:30.565524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:30.567292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 997/1982] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:30 2022] DELETE /identity/v3/users/acb608285a464b1db9816369d6e76f92 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:30.577343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 986/1983] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:30.582010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:30.583814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:30.620594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:30.653269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e471a94-bdd2-4fc7-8f5b-3455ed329b6e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:30.658504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d2fc-ab1a-4d59-b219-649c01361ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:30.664927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d2fc-ab1a-4d59-b219-649c01361ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:30.664927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d2fc-ab1a-4d59-b219-649c01361ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:30.664927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d2fc-ab1a-4d59-b219-649c01361ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:30.664927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42d2fc-ab1a-4d59-b219-649c01361ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:30.664927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42d2fc-ab1a-4d59-b219-649c01361ceb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:30.674649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 998/1984] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:30.681130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:30.691159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791510bb-4a19-4af1-aa4c-19c7f7bbf83e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:30.696063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:30.699337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:30.699337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:30.699337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] PATH_INFO: `/v3/projects/27127e7e98f14df4bfadcd0bd8bcf7a3/users/319323130d2a4e4ea7e4cff4c0d6f0f4/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:30.712442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27127e7e98f14df4bfadcd0bd8bcf7a3, user_id=319323130d2a4e4ea7e4cff4c0d6f0f4, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:30.715406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:30.827533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 987/1985] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:30 2022] PUT /identity/v3/projects/27127e7e98f14df4bfadcd0bd8bcf7a3/users/319323130d2a4e4ea7e4cff4c0d6f0f4/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:30.839320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:30.839611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:30.840250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:30.871705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] MFA Rules not processed for user `319323130d2a4e4ea7e4cff4c0d6f0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:30.872944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:30.938289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 999/1986] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:30.942962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:30.958353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:30.959892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.017729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17e541-b4e0-402c-8744-ddd26b9b0e8c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.024178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.024911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.025196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.025463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.026173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.028066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.033731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1d932-cd3f-4f4f-8c8c-8dec8281f4df None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.039060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.047305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.047305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.047305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] PATH_INFO: `/v3/projects/0ae0a809de8141fbafa9f3881447361d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.047305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae0a809de8141fbafa9f3881447361d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.047305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.123375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:31.130169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 988/1987] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:30 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:31.131018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.132617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.189109 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae0a809de8141fbafa9f3881447361d', 'request_id': 'req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.198784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1b506-6572-4291-80b4-68adc486d16c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.255640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.256165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.256433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.257653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] PATH_INFO: `/v3/projects/a36c55a6288549e6ac718f3d5370d4b8` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.258525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36c55a6288549e6ac718f3d5370d4b8)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.260433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.263472 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae0a809de8141fbafa9f3881447361d', 'request_id': 'req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.272760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae0a809de8141fbafa9f3881447361d', 'request_id': 'req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.286728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] The token cache is being invalidate because project 0ae0a809de8141fbafa9f3881447361d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:31.287014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.288376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.289986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1000/1988] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:30 2022] DELETE /identity/v3/projects/0ae0a809de8141fbafa9f3881447361d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:31.290911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.292294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.305189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7ddd-ef59-4e79-b05a-f2a4101ce4be admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.313220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d583f-144f-4623-898e-7a7d724f7ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.313940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d583f-144f-4623-898e-7a7d724f7ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.314912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d583f-144f-4623-898e-7a7d724f7ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.315193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d583f-144f-4623-898e-7a7d724f7ddc admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.315827 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d583f-144f-4623-898e-7a7d724f7ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.317752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d583f-144f-4623-898e-7a7d724f7ddc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.388824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36c55a6288549e6ac718f3d5370d4b8', 'request_id': 'req-690c72dc-e993-432b-903e-e6a3fa6f2109'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.405698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1001/1989] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:31.405698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f4c9-ad7e-4470-a5be-2caac070c339 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.405698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f4c9-ad7e-4470-a5be-2caac070c339 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.405698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4f4c9-ad7e-4470-a5be-2caac070c339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.424054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1dd4f4c9-ad7e-4470-a5be-2caac070c339 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:31.425274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1dd4f4c9-ad7e-4470-a5be-2caac070c339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.440151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36c55a6288549e6ac718f3d5370d4b8', 'request_id': 'req-690c72dc-e993-432b-903e-e6a3fa6f2109'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.448451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36c55a6288549e6ac718f3d5370d4b8', 'request_id': 'req-690c72dc-e993-432b-903e-e6a3fa6f2109'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.458883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] The token cache is being invalidate because project a36c55a6288549e6ac718f3d5370d4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:31.459172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.459842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:31.462777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 989/1990] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:31 2022] DELETE /identity/v3/projects/a36c55a6288549e6ac718f3d5370d4b8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:31.463590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.465138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.485881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1002/1991] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:31.487739 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41152bd-e0c8-4d35-99cf-1719b0675547 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.488186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41152bd-e0c8-4d35-99cf-1719b0675547 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.488600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41152bd-e0c8-4d35-99cf-1719b0675547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.508614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e41152bd-e0c8-4d35-99cf-1719b0675547 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:31.510066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e41152bd-e0c8-4d35-99cf-1719b0675547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.524559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1003/1992] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:31.524559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661febd-9269-452f-947f-ef93af85b0dd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.524559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661febd-9269-452f-947f-ef93af85b0dd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.524559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661febd-9269-452f-947f-ef93af85b0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.529391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c72dc-e993-432b-903e-e6a3fa6f2109 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.534300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534d6bf-2f95-4109-b6e1-44b1ad1a4d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.534914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534d6bf-2f95-4109-b6e1-44b1ad1a4d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.535197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534d6bf-2f95-4109-b6e1-44b1ad1a4d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.535478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534d6bf-2f95-4109-b6e1-44b1ad1a4d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.536130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534d6bf-2f95-4109-b6e1-44b1ad1a4d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.537852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534d6bf-2f95-4109-b6e1-44b1ad1a4d1c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.542274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a661febd-9269-452f-947f-ef93af85b0dd None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:31.542274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a661febd-9269-452f-947f-ef93af85b0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.548204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 990/1993] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:31.549205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1004/1994] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:31.549451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d497e9d-6c97-4eb3-a039-7c75f5e5f2b6 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.549771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d497e9d-6c97-4eb3-a039-7c75f5e5f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.550025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d497e9d-6c97-4eb3-a039-7c75f5e5f2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.552330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661febd-9269-452f-947f-ef93af85b0dd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.562308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661febd-9269-452f-947f-ef93af85b0dd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.566612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:31.567123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.567780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.568147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.568351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] PATH_INFO: `/v3/projects/fa12d447b30f4ff58464c839a1d067b4/users/b902b885f9114946ac324782769d8c56/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.572576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5d497e9d-6c97-4eb3-a039-7c75f5e5f2b6 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:31.573791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5d497e9d-6c97-4eb3-a039-7c75f5e5f2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:31.582060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 991/1995] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:31.582859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d497e9d-6c97-4eb3-a039-7c75f5e5f2b6 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.584644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa12d447b30f4ff58464c839a1d067b4, user_id=b902b885f9114946ac324782769d8c56, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.588490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.592975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d497e9d-6c97-4eb3-a039-7c75f5e5f2b6 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.597808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.598416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.598696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.598972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.599621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:31.599928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.601715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.613205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 992/1996] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:31.614025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.623982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277b8bd-f3a2-42d2-9455-cb13e09eae68 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.628539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.629131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.629409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.629674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.630344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:31.630635 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.632772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.646028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 993/1997] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:31.646896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.656729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd9aea-ae48-4fe8-ba72-5d8caedf20df admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.661211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.661820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.662100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.662371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.663019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.665052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.672322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1005/1998] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:31 2022] PUT /identity/v3/projects/fa12d447b30f4ff58464c839a1d067b4/users/b902b885f9114946ac324782769d8c56/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:31.677503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.738517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651266b2-d285-43b1-ad1f-56a5d44e3fcf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.743846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.744438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.744722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.745002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.745633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.747572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.764857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 994/1999] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:31.765707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.825575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa94980-c76b-48ca-9e81-14fb3cf84085 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.830244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.830849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.831131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.831409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.832050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.833763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.845889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 995/2000] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:31.851852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1006/2001] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:31.851990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.852120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:31.859780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207a386-3db0-460d-b740-92b00e999b23 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.917043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d195ad-f14c-490b-9b72-0096d7ba6de5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:31.917330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ec23f-9d49-4072-bc47-9320ec9c7043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.917882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec23f-9d49-4072-bc47-9320ec9c7043 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.918157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec23f-9d49-4072-bc47-9320ec9c7043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.918429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ec23f-9d49-4072-bc47-9320ec9c7043 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.919051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ec23f-9d49-4072-bc47-9320ec9c7043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.920165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:31.920826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:31.921136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:31.921229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ec23f-9d49-4072-bc47-9320ec9c7043 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.921416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] PATH_INFO: `/v3/projects/fa12d447b30f4ff58464c839a1d067b4/users/b902b885f9114946ac324782769d8c56/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:31.923460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa12d447b30f4ff58464c839a1d067b4, user_id=b902b885f9114946ac324782769d8c56, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:31.926098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:31.937563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1007/2002] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:31 2022] PUT /identity/v3/projects/fa12d447b30f4ff58464c839a1d067b4/users/b902b885f9114946ac324782769d8c56/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:31.947971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.008871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cbe2e-d67f-4f64-9b7c-bc588ec90b22 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.013662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.017690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.017690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.017690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.017690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:32.017690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.019150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.217531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 996/2003] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/users => generated 295 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:32.217531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c992a-37b9-4c4d-9756-e21d0a7f312a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.217531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c992a-37b9-4c4d-9756-e21d0a7f312a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.217531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c992a-37b9-4c4d-9756-e21d0a7f312a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.243871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1008/2004] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:31 2022] POST /identity/v3/users => generated 290 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:32.245034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.247938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-833c992a-37b9-4c4d-9756-e21d0a7f312a None None] MFA Rules not processed for user `b902b885f9114946ac324782769d8c56`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:32.249166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-833c992a-37b9-4c4d-9756-e21d0a7f312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:32.254525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ee02-0f04-435a-a1bd-db9d718b8188 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.259403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.260080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.260363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.260635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.261272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.262871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.273422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1009/2005] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:32.274185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.283899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5dedd-f4da-4590-bcc9-f7693ed18f52 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.289459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.290086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.290361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.290631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.291265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.292876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.303709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1010/2006] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:32.305365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.314643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca11a3-a977-4bfa-a979-83b228cfbbfe admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.319243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 997/2007] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:32.319399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.320576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c992a-37b9-4c4d-9756-e21d0a7f312a None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.323504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.323504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.323504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.324031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.325687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.341888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c992a-37b9-4c4d-9756-e21d0a7f312a None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.345650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1011/2008] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:32.345650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.346752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:32.350217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.351758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.352196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.352628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.353425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.355194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.359584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:32.368070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 998/2009] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:32.369052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.379345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f63acb-f55e-472e-99f0-0e10458d69b1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.385013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496742f-b2fb-4d27-bae6-725ab93b1b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.385681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496742f-b2fb-4d27-bae6-725ab93b1b60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.385984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496742f-b2fb-4d27-bae6-725ab93b1b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.386268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496742f-b2fb-4d27-bae6-725ab93b1b60 admin admin] PATH_INFO: `/v3/projects/383a9659fe374bfebc2898477b1ac8e9/users/d410a3a123db41c9bb5099ff0e37ccf9/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.402574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496742f-b2fb-4d27-bae6-725ab93b1b60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383a9659fe374bfebc2898477b1ac8e9, user_id=d410a3a123db41c9bb5099ff0e37ccf9, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.405260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496742f-b2fb-4d27-bae6-725ab93b1b60 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.415846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b09ae-660a-4886-9bc1-d0172986de31 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.419927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.420545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.420828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.421555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.430993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.432448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.442277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1012/2010] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:32.448586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.454033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27ee8-2a42-44d9-8865-d30050845c95 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.462030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.462854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.463235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.463615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] PATH_INFO: `/v3/projects/3b727e42953c4fd195587bb7eeedfb60/users/6fc3d35d21f84324aeade5e2678f1ed0/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.482250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b727e42953c4fd195587bb7eeedfb60, user_id=6fc3d35d21f84324aeade5e2678f1ed0, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.485604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.498028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 999/2011] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:32 2022] PUT /identity/v3/projects/383a9659fe374bfebc2898477b1ac8e9/users/d410a3a123db41c9bb5099ff0e37ccf9/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:32.504966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4347ed8-bf7a-4309-9779-c8e526ae7343 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.504966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4347ed8-bf7a-4309-9779-c8e526ae7343 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.504966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4347ed8-bf7a-4309-9779-c8e526ae7343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.511928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:32.533673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d4347ed8-bf7a-4309-9779-c8e526ae7343 None None] MFA Rules not processed for user `d410a3a123db41c9bb5099ff0e37ccf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:32.534970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d4347ed8-bf7a-4309-9779-c8e526ae7343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:32.577601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1013/2012] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:32 2022] PUT /identity/v3/projects/3b727e42953c4fd195587bb7eeedfb60/users/6fc3d35d21f84324aeade5e2678f1ed0/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:32.578465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.602080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1000/2013] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:32.610807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ce6c-d4b0-470a-8cd4-28419ef7cadf None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.610807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ce6c-d4b0-470a-8cd4-28419ef7cadf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.610807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365ce6c-d4b0-470a-8cd4-28419ef7cadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.633518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5365ce6c-d4b0-470a-8cd4-28419ef7cadf None None] MFA Rules not processed for user `6fc3d35d21f84324aeade5e2678f1ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:32.635030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5365ce6c-d4b0-470a-8cd4-28419ef7cadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:32.640511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454deb7-5b0a-440e-bca0-cda78cfcfc49 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.644046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.644713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.644995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.645266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.647331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:32.700571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1001/2014] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:32.701490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365ce6c-d4b0-470a-8cd4-28419ef7cadf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.706995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.708696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.721731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1014/2015] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:32.722541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.733794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:32.766870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365ce6c-d4b0-470a-8cd4-28419ef7cadf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.772357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.773011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.773290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.773587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.774203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.776143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.790124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0a66b-e3ac-4b93-a38e-93d0cae7eb37 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.796732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:32.797691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.798407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.798688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.798962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.799640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.801559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.890125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1002/2016] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:32 2022] POST /identity/v3/projects => generated 389 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:32.894264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1015/2017] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:32 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:32.894415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.895171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:32.958302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660dd97-42d5-4403-acfa-b5c9afb9f9fb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.958546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffb66-015b-4205-95d2-4d3cbe8ee503 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:32.963173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.963320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:32.963783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.963904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:32.964051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.964144 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:32.964319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.964418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:32.964993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.965142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:32.966961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:32.967588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.048679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1003/2018] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:32 2022] POST /identity/v3/users => generated 301 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:33.052477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.062832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cc598-9513-4a93-bfbf-9636726d3555 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.068746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:33.070376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.071028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.071364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.071636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.075282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1016/2019] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:32 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:33.075434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.075434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.076911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.084602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1004/2020] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:33.088379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97790-0882-43a4-8fbf-c5c95d87f6d8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.089643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.093377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.094017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.094898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.094898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.095227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.096817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.099813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b3e07-5285-4463-a3f9-0d46818850f6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.105104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.105870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.106149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.106478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] PATH_INFO: `/v3/projects/bc8ee53c69b3457ba5e6333c0595cfcd/users/be57dbf407e84af6b04b9f6cd331bbc1/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.108332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1017/2021] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:33.112356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.121752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8ee53c69b3457ba5e6333c0595cfcd, user_id=be57dbf407e84af6b04b9f6cd331bbc1, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.122360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b155f-be9b-4ad8-8bc4-2f4d75a8d6f7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.124380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.127123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.127773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.128057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.128331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] PATH_INFO: `/v3/projects/a8f6c3f2754d423faa83c17d132689e3/users/51ae6e6ebe164c109150da1a8ca0b0dc/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.142815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f6c3f2754d423faa83c17d132689e3, user_id=51ae6e6ebe164c109150da1a8ca0b0dc, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.145756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.150298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:33.203244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1005/2022] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:33 2022] PUT /identity/v3/projects/bc8ee53c69b3457ba5e6333c0595cfcd/users/be57dbf407e84af6b04b9f6cd331bbc1/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:33.207529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.248557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1018/2023] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:33 2022] PUT /identity/v3/projects/a8f6c3f2754d423faa83c17d132689e3/users/51ae6e6ebe164c109150da1a8ca0b0dc/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:33.252374 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.276214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fde0d8-9210-48c9-8b8d-580f271ea05f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.319706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48ef14-c808-411c-be31-807f07ef3ecd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.324136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.324749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.325027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.325303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.325974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.327561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.336251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.339131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.339131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.339131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.339131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.340159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1019/2024] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:33.340258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.343400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.350342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1006/2025] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:33.353569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a7a58-c6d4-431a-95eb-56433f56abb6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.354679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.359474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc315af-01ff-471f-b5a2-e29f3beebc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.364201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc315af-01ff-471f-b5a2-e29f3beebc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.364201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc315af-01ff-471f-b5a2-e29f3beebc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.364201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc315af-01ff-471f-b5a2-e29f3beebc7b admin admin] PATH_INFO: `/v3/projects/a8f6c3f2754d423faa83c17d132689e3/users/51ae6e6ebe164c109150da1a8ca0b0dc/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.364201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc315af-01ff-471f-b5a2-e29f3beebc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f6c3f2754d423faa83c17d132689e3, user_id=51ae6e6ebe164c109150da1a8ca0b0dc, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.364553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647218f-4eba-4a3c-8b1a-73fcf1e758da admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.365516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc315af-01ff-471f-b5a2-e29f3beebc7b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.370234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0ae04-d5bd-4896-aeab-dd993d682cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.370958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0ae04-d5bd-4896-aeab-dd993d682cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.371293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0ae04-d5bd-4896-aeab-dd993d682cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.371501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0ae04-d5bd-4896-aeab-dd993d682cf5 admin admin] PATH_INFO: `/v3/projects/bc8ee53c69b3457ba5e6333c0595cfcd/users/be57dbf407e84af6b04b9f6cd331bbc1/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.373461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0ae04-d5bd-4896-aeab-dd993d682cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8ee53c69b3457ba5e6333c0595cfcd, user_id=be57dbf407e84af6b04b9f6cd331bbc1, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.376526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0ae04-d5bd-4896-aeab-dd993d682cf5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.393587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1020/2026] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:33 2022] PUT /identity/v3/projects/a8f6c3f2754d423faa83c17d132689e3/users/51ae6e6ebe164c109150da1a8ca0b0dc/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:33.396500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1007/2027] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:33 2022] PUT /identity/v3/projects/bc8ee53c69b3457ba5e6333c0595cfcd/users/be57dbf407e84af6b04b9f6cd331bbc1/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:33.405816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bf987-84b2-49c5-b549-5431395580b7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.406270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bf987-84b2-49c5-b549-5431395580b7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.407823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.407963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bf987-84b2-49c5-b549-5431395580b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.408237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.408582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.413134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:33.444525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] MFA Rules not processed for user `be57dbf407e84af6b04b9f6cd331bbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:33.445262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fc0bf987-84b2-49c5-b549-5431395580b7 None None] MFA Rules not processed for user `51ae6e6ebe164c109150da1a8ca0b0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:33.446494 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc0bf987-84b2-49c5-b549-5431395580b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:33.447037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:33.530103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1021/2028] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:33.537908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1008/2029] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:33.539320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bf987-84b2-49c5-b549-5431395580b7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.549590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.552787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:33.606461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bf987-84b2-49c5-b549-5431395580b7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.611772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.613272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.614267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.614560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.618616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cdc7a-1d08-43eb-a516-1befe301a501 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.622553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.623414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.623791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.624146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.624797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.626265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.636020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.637680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.645582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1022/2030] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:33.653634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1009/2031] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:33 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:33.793443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.802421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:33.857183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9a550-f585-40c7-aee2-3da9c229c5a5 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.861653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.862303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.862580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.862851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] PATH_INFO: `/v3/users/d410a3a123db41c9bb5099ff0e37ccf9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.863971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d410a3a123db41c9bb5099ff0e37ccf9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.866027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.891211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:33.896308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:33.900118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33eae-3c06-45d7-a5e2-f2d2d6fa7faf service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:33.903768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81deb7b-3d4b-4bca-9228-c59f2f0289a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:33.904364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81deb7b-3d4b-4bca-9228-c59f2f0289a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:33.904647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81deb7b-3d4b-4bca-9228-c59f2f0289a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:33.904919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81deb7b-3d4b-4bca-9228-c59f2f0289a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:33.961566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81deb7b-3d4b-4bca-9228-c59f2f0289a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:33.963180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81deb7b-3d4b-4bca-9228-c59f2f0289a3 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:33.971993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1010/2032] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:34.049130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd410a3a123db41c9bb5099ff0e37ccf9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:34.049809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd410a3a123db41c9bb5099ff0e37ccf9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:34.056771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd410a3a123db41c9bb5099ff0e37ccf9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:34.068714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd410a3a123db41c9bb5099ff0e37ccf9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:34.069372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd410a3a123db41c9bb5099ff0e37ccf9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:34.154962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1023/2033] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:33 2022] DELETE /identity/v3/users/d410a3a123db41c9bb5099ff0e37ccf9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:34.159565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:34.159852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:34.160130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:34.180385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:34.181866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:34.243062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1011/2034] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:34.252977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:34.254527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:34.312664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06c264-8f0d-4dac-971a-da82d03aa8ca admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:34.316368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:34.316978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:34.317259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:34.317531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:34.327142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:34.328570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:34.338637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1024/2035] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:34.791933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:34.793554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:34.807757 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fdae8-03f8-4122-add4-34a92a54f4e7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:34.813274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:34.813980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:34.814415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:34.814755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] PATH_INFO: `/v3/projects/383a9659fe374bfebc2898477b1ac8e9` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:34.816196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383a9659fe374bfebc2898477b1ac8e9)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:34.818435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:34.950849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383a9659fe374bfebc2898477b1ac8e9', 'request_id': 'req-98c968d8-969c-4d1a-a5fe-888adf70aa50'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.011831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383a9659fe374bfebc2898477b1ac8e9', 'request_id': 'req-98c968d8-969c-4d1a-a5fe-888adf70aa50'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.018652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383a9659fe374bfebc2898477b1ac8e9', 'request_id': 'req-98c968d8-969c-4d1a-a5fe-888adf70aa50'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.029456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] The token cache is being invalidate because project 383a9659fe374bfebc2898477b1ac8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:35.029770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.030386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.031852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1012/2036] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:34 2022] DELETE /identity/v3/projects/383a9659fe374bfebc2898477b1ac8e9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:35.035920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:35.045568 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7024a4-8ddb-4f5c-bdbd-b589f587e1da service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:35.071914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:35.103162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:35.104695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:35.104695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:35.104695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] PATH_INFO: `/v3/users/be57dbf407e84af6b04b9f6cd331bbc1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:35.105499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be57dbf407e84af6b04b9f6cd331bbc1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:35.107432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:35.290452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be57dbf407e84af6b04b9f6cd331bbc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.291095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be57dbf407e84af6b04b9f6cd331bbc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.299454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be57dbf407e84af6b04b9f6cd331bbc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.311819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be57dbf407e84af6b04b9f6cd331bbc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.312512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be57dbf407e84af6b04b9f6cd331bbc1'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:35.403393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1025/2037] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:35 2022] DELETE /identity/v3/users/be57dbf407e84af6b04b9f6cd331bbc1 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:35.824363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:35.826165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:35.840451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c968d8-969c-4d1a-a5fe-888adf70aa50 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:35.868373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:35.899055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:35.899687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:35.899968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:35.900241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] PATH_INFO: `/v3/projects/bc8ee53c69b3457ba5e6333c0595cfcd` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:35.901368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8ee53c69b3457ba5e6333c0595cfcd)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:35.903315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:35.966921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:35.968449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:35.982263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e288f6-a71e-4846-87ce-daf491549ed0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:35.987197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:35.987837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:35.988509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:35.988509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] PATH_INFO: `/v3/users/319323130d2a4e4ea7e4cff4c0d6f0f4` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:35.989515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=319323130d2a4e4ea7e4cff4c0d6f0f4)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:35.992109 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.043829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8ee53c69b3457ba5e6333c0595cfcd', 'request_id': 'req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.134378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8ee53c69b3457ba5e6333c0595cfcd', 'request_id': 'req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.142077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8ee53c69b3457ba5e6333c0595cfcd', 'request_id': 'req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.156777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] The token cache is being invalidate because project bc8ee53c69b3457ba5e6333c0595cfcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:36.157094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.157792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-1e3f6ab4-db81-43c9-aa83-05efe2fc5fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.159335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1013/2038] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:35 2022] DELETE /identity/v3/projects/bc8ee53c69b3457ba5e6333c0595cfcd => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:36.164859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a9206-1ea0-48bf-82a2-a82c81bca7ba None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.165143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a9206-1ea0-48bf-82a2-a82c81bca7ba None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.165415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a9206-1ea0-48bf-82a2-a82c81bca7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.186025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-300a9206-1ea0-48bf-82a2-a82c81bca7ba None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:36.187240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-300a9206-1ea0-48bf-82a2-a82c81bca7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:36.248038 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1014/2039] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:36.254532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319323130d2a4e4ea7e4cff4c0d6f0f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.255732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319323130d2a4e4ea7e4cff4c0d6f0f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.256075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2830d-213f-44a9-a75c-34dfc42f5633 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.256361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2830d-213f-44a9-a75c-34dfc42f5633 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.256631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2830d-213f-44a9-a75c-34dfc42f5633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.265024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319323130d2a4e4ea7e4cff4c0d6f0f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.276609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-30a2830d-213f-44a9-a75c-34dfc42f5633 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:36.277769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319323130d2a4e4ea7e4cff4c0d6f0f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.277891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-30a2830d-213f-44a9-a75c-34dfc42f5633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:36.278459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-6467ebe6-a78d-4ea1-a2ce-7d9dc36c7496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319323130d2a4e4ea7e4cff4c0d6f0f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:36.285833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1015/2040] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:36.293850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2830d-213f-44a9-a75c-34dfc42f5633 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:36.303275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2830d-213f-44a9-a75c-34dfc42f5633 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:36.308241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:36.308877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.309148 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.309414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.310087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:36.310647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:36.311970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.324157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1016/2041] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:06:36.328528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:36.339439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c06218-1eca-4465-bf77-f099fd52353e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:36.344097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:36.344691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.344973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.345245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.345947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:36.347878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.360537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1026/2042] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:35 2022] DELETE /identity/v3/users/319323130d2a4e4ea7e4cff4c0d6f0f4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:36.364780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.365159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.365528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.384976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:36.386131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:36.399026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:36.442034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1017/2043] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:36 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:36.478982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:36.479339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1027/2044] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:36.479339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:36.479339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:36.479575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:36.509316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7226b-21d5-47e2-85ca-91aab998b90f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:36.514164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:36.514774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.514871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ca57e-dfb7-424b-bb53-3975d4c4f30e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:36.515055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.515329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.515962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:36.517908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.564456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:36.565091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.565373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.565649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.575496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:36.577039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.586997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1028/2045] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:36.629117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1018/2046] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:36 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:36.633868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:36.650868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14637ed9-3ffa-49a3-9b13-e50dbc056316 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:36.656269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:36.656873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.657149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.657421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.658089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:36.659733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.670383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1029/2047] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:36.674484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:36.684735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede21a6-13a4-4862-a024-834c4b306627 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:36.689329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e73fe3-5d4c-4c4c-a8d7-0480fe2ceef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:36.690049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e73fe3-5d4c-4c4c-a8d7-0480fe2ceef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.690346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e73fe3-5d4c-4c4c-a8d7-0480fe2ceef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.690617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e73fe3-5d4c-4c4c-a8d7-0480fe2ceef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.691341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e73fe3-5d4c-4c4c-a8d7-0480fe2ceef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:36.692817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e73fe3-5d4c-4c4c-a8d7-0480fe2ceef2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.703091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1019/2048] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:36.707200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:36.717816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6151d-f85a-4f1f-9222-e5474bebc02a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:36.722424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:36.723065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.723347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.723644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] PATH_INFO: `/v3/projects/4dc826493dc049d791cec8b4bb95943f/users/e643c21b8acc4f098a0ca6a2865a9fc9/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.737898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc826493dc049d791cec8b4bb95943f, user_id=e643c21b8acc4f098a0ca6a2865a9fc9, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:36.740612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:36.842005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1030/2049] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:36 2022] PUT /identity/v3/projects/4dc826493dc049d791cec8b4bb95943f/users/e643c21b8acc4f098a0ca6a2865a9fc9/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:36.854142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:36.854654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:36.854743 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:36.885759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] MFA Rules not processed for user `e643c21b8acc4f098a0ca6a2865a9fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:36.886997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:36.957242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1020/2050] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:36.962549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.010557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:37.026208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a3bd-c76d-443f-890c-ad1344ba975b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.031070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.035455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.035455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.035455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.035455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.035455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.062196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.069033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:37.080337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2a05c-bb10-4ba6-a5d4-1121f845bfc5 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.086010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.086690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.087501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.087791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] PATH_INFO: `/v3/projects/27127e7e98f14df4bfadcd0bd8bcf7a3` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.088960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27127e7e98f14df4bfadcd0bd8bcf7a3)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.090907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.208893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1031/2051] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:36 2022] POST /identity/v3/projects => generated 387 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:37.208893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.246457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27127e7e98f14df4bfadcd0bd8bcf7a3', 'request_id': 'req-492fbe40-ff30-45f8-bb14-9c83eedf5585'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.268791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8deb34-efec-4331-baae-c8744e382c2e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.317096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27127e7e98f14df4bfadcd0bd8bcf7a3', 'request_id': 'req-492fbe40-ff30-45f8-bb14-9c83eedf5585'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.324648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27127e7e98f14df4bfadcd0bd8bcf7a3', 'request_id': 'req-492fbe40-ff30-45f8-bb14-9c83eedf5585'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.329115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.329841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.330188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.330537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.331239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.333207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.337468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] The token cache is being invalidate because project 27127e7e98f14df4bfadcd0bd8bcf7a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:37.344059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.344059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.344059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1021/2052] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:37 2022] DELETE /identity/v3/projects/27127e7e98f14df4bfadcd0bd8bcf7a3 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:37.346855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.348441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:37.363797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fbe40-ff30-45f8-bb14-9c83eedf5585 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.368013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.378519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.378519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.378519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] PATH_INFO: `/v3/users/b902b885f9114946ac324782769d8c56` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.378519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b902b885f9114946ac324782769d8c56)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.378519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.378519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:37.438054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1032/2053] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:37 2022] POST /identity/v3/users => generated 300 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:37.441968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.443658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:37.455489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda90ded-4fb5-46ff-9642-98a441e83bdc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.461807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.462551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.462967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.463140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.463804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.465411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.479764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1033/2054] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:37.480733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.490428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cfd284-6002-4536-ae4a-c4a96bc8cff2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.496430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.497101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.497368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.497677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] PATH_INFO: `/v3/projects/a2dfe2e996f44a4694c14d7ffc7a7479/users/7f8371913e3c45ff94ef21b0734d11fa/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.514114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2dfe2e996f44a4694c14d7ffc7a7479, user_id=7f8371913e3c45ff94ef21b0734d11fa, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.516937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.557571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b902b885f9114946ac324782769d8c56'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.558334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b902b885f9114946ac324782769d8c56'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.565281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b902b885f9114946ac324782769d8c56'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.577117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b902b885f9114946ac324782769d8c56'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.577791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b902b885f9114946ac324782769d8c56'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:37.637828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1034/2055] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:37 2022] PUT /identity/v3/projects/a2dfe2e996f44a4694c14d7ffc7a7479/users/7f8371913e3c45ff94ef21b0734d11fa/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:37.641584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.643261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:37.678492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:37.694362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1022/2056] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:37 2022] DELETE /identity/v3/users/b902b885f9114946ac324782769d8c56 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:37.711832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b38e3a-5289-404e-a2c3-d99f82eaa9fe admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.769528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9142fb7-390b-41ae-9ddc-ca1687b6b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.770600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142fb7-390b-41ae-9ddc-ca1687b6b22a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.771040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142fb7-390b-41ae-9ddc-ca1687b6b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.771319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9142fb7-390b-41ae-9ddc-ca1687b6b22a admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.771984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9142fb7-390b-41ae-9ddc-ca1687b6b22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.774431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9142fb7-390b-41ae-9ddc-ca1687b6b22a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.786025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1035/2057] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:37.790887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.801327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491078fb-3463-469f-81f0-849f50e2d18b admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:37.806671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:37.807326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.807610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.807886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] PATH_INFO: `/v3/projects/a2dfe2e996f44a4694c14d7ffc7a7479/users/7f8371913e3c45ff94ef21b0734d11fa/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.809797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2dfe2e996f44a4694c14d7ffc7a7479, user_id=7f8371913e3c45ff94ef21b0734d11fa, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:37.812442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:37.823456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1023/2058] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:37 2022] PUT /identity/v3/projects/a2dfe2e996f44a4694c14d7ffc7a7479/users/7f8371913e3c45ff94ef21b0734d11fa/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:37.833792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:37.834085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:37.834380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:37.864298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] MFA Rules not processed for user `7f8371913e3c45ff94ef21b0734d11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:37.865555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:37.936959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1036/2059] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:37.950872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:37.951826 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:38.023820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4db8bf-9569-47ed-8bbf-7ccd9661b7b9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.027879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67ffff-6004-4f3f-9d24-9f97df877e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.028816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67ffff-6004-4f3f-9d24-9f97df877e10 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.028816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67ffff-6004-4f3f-9d24-9f97df877e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.031437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67ffff-6004-4f3f-9d24-9f97df877e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.038795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67ffff-6004-4f3f-9d24-9f97df877e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:38.040276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67ffff-6004-4f3f-9d24-9f97df877e10 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:38.050873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1024/2060] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:38.130214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:38.132587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:38.203755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59628ffe-62be-4862-91f6-66a7b7f8af19 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.208161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.208796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.209078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.209350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] PATH_INFO: `/v3/projects/fa12d447b30f4ff58464c839a1d067b4` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.211240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa12d447b30f4ff58464c839a1d067b4)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:38.213139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:38.214893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:38.354259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa12d447b30f4ff58464c839a1d067b4', 'request_id': 'req-fbd20cac-2b27-40ee-ace8-1bc22d1981de'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:38.419789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa12d447b30f4ff58464c839a1d067b4', 'request_id': 'req-fbd20cac-2b27-40ee-ace8-1bc22d1981de'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:38.426778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa12d447b30f4ff58464c839a1d067b4', 'request_id': 'req-fbd20cac-2b27-40ee-ace8-1bc22d1981de'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:38.437938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] The token cache is being invalidate because project fa12d447b30f4ff58464c839a1d067b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:38.441113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:38.441113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-fbd20cac-2b27-40ee-ace8-1bc22d1981de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:38.441113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1037/2061] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:38 2022] DELETE /identity/v3/projects/fa12d447b30f4ff58464c839a1d067b4 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:38.445908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b802c-3da3-46f4-9ccc-23375141c0d7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.446190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b802c-3da3-46f4-9ccc-23375141c0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.446485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b802c-3da3-46f4-9ccc-23375141c0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.466918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0b8b802c-3da3-46f4-9ccc-23375141c0d7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:38.468137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0b8b802c-3da3-46f4-9ccc-23375141c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:38.534242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1025/2062] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:38.542309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16054df1-9a6b-4919-92f1-02d0abf4817a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.542599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16054df1-9a6b-4919-92f1-02d0abf4817a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.542872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16054df1-9a6b-4919-92f1-02d0abf4817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.563137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-16054df1-9a6b-4919-92f1-02d0abf4817a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:38.564340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-16054df1-9a6b-4919-92f1-02d0abf4817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:38.573694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1038/2063] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:38.578231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b802c-3da3-46f4-9ccc-23375141c0d7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:38.588515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b802c-3da3-46f4-9ccc-23375141c0d7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.593447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.594087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.594367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.594640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.595368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:38.595676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:38.597281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:38.599657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:38.609585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1026/2064] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:38.619643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16054df1-9a6b-4919-92f1-02d0abf4817a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:38.627335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16054df1-9a6b-4919-92f1-02d0abf4817a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.633656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.634410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.634698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.635001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.635634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:38.637620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:38.755426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1039/2065] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:38 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:38.757195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:38.770838 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25061717-7a3f-494b-b396-ab9481402d59 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.828056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.828670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.828951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.829225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.829892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:38.831839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:38.893265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1027/2066] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:38 2022] POST /identity/v3/users => generated 284 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:38.899601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:38.910887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2234dbb-4875-4e5b-a6a8-90d87c303dc9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.916248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.916896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.917152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.917422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.918188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:38.919995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:38.931096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1040/2067] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:38.935062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:38.945099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a0f13-103a-4c13-bd79-fabec7663279 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.950696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10981c1-0e3b-4ec2-ba1a-357833faabfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.951293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10981c1-0e3b-4ec2-ba1a-357833faabfd admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.951842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10981c1-0e3b-4ec2-ba1a-357833faabfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.951842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10981c1-0e3b-4ec2-ba1a-357833faabfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:38.952455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10981c1-0e3b-4ec2-ba1a-357833faabfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:38.954423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10981c1-0e3b-4ec2-ba1a-357833faabfd admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:38.965449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1028/2068] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:38.969560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:38.980310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d65e2-8ebb-4a44-8d83-21791c4cef75 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:38.984937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:38.985585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:38.985900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:38.986177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] PATH_INFO: `/v3/projects/4fd30b71c7ee4e859079e7282a84bf98/users/b909f8a5a75c482c81d493e029b99095/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.004106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd30b71c7ee4e859079e7282a84bf98, user_id=b909f8a5a75c482c81d493e029b99095, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.006891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.105128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1041/2069] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:38 2022] PUT /identity/v3/projects/4fd30b71c7ee4e859079e7282a84bf98/users/b909f8a5a75c482c81d493e029b99095/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:39.115252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.115541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.115816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.143840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] MFA Rules not processed for user `b909f8a5a75c482c81d493e029b99095`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:39.145059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:39.209537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1029/2070] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:39.214516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.264480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.265998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:39.279438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda342e-b1db-4648-a3ce-b6fb278e2805 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.284168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.284768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.285044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.285314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.285969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.287873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.329434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c7216-4253-44c6-b54c-cd8a80c4a238 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.333969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.334575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.334849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.335731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] PATH_INFO: `/v3/users/e643c21b8acc4f098a0ca6a2865a9fc9` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.336987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e643c21b8acc4f098a0ca6a2865a9fc9)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.338915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.343383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:39.375355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1042/2071] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:39 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:39.379829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.391800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:39.440689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cad94-7eeb-4e1c-9068-a5bc95c8cc06 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.446020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.446696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.446930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.447206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.447841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.449950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.565862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e643c21b8acc4f098a0ca6a2865a9fc9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:39.566621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e643c21b8acc4f098a0ca6a2865a9fc9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:39.573012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1043/2072] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:39 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:39.575039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.576180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e643c21b8acc4f098a0ca6a2865a9fc9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:39.583388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:39.589416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e643c21b8acc4f098a0ca6a2865a9fc9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:39.590480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0027086f-04dd-4039-b039-a6bdc084772a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e643c21b8acc4f098a0ca6a2865a9fc9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:39.598204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7ea2-f246-422f-a9d9-c28965b95167 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.602867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.607101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.607101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.607101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.607101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.607101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.616735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1044/2073] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:39.620878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.630045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81044fc-ecf4-4fe6-af18-01031e0a2314 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.635151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.639757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.639757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.639757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] PATH_INFO: `/v3/projects/1718fd7e002f409cb785964dc96a4465/users/2047b4eacb064fe29f344c96cf7038d2/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.651610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1718fd7e002f409cb785964dc96a4465, user_id=2047b4eacb064fe29f344c96cf7038d2, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.654262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.702920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1030/2074] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:39 2022] DELETE /identity/v3/users/e643c21b8acc4f098a0ca6a2865a9fc9 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:39.707356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.707644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.707922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.727600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:39.728836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:39.745593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1045/2075] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:39 2022] PUT /identity/v3/projects/1718fd7e002f409cb785964dc96a4465/users/2047b4eacb064fe29f344c96cf7038d2/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:39.749519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.793520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1031/2076] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:39.803447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.805480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:39.815163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77169757-4ace-4be8-aeb2-f944b63722a6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.821422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.822093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.822373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.822643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.823286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.824898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.835957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1046/2077] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:39.840106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:39.849769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97610531-f9e1-456c-ba1f-642dd480eec2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.855032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6986f-46a7-482d-982b-ca631d1d818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.855684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6986f-46a7-482d-982b-ca631d1d818c admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.855966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6986f-46a7-482d-982b-ca631d1d818c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.856237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6986f-46a7-482d-982b-ca631d1d818c admin admin] PATH_INFO: `/v3/projects/1718fd7e002f409cb785964dc96a4465/users/2047b4eacb064fe29f344c96cf7038d2/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.858184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6986f-46a7-482d-982b-ca631d1d818c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1718fd7e002f409cb785964dc96a4465, user_id=2047b4eacb064fe29f344c96cf7038d2, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.860771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6986f-46a7-482d-982b-ca631d1d818c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.871615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3bf31-be15-4120-a3c5-1ec9cf0dfe9b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:39.872727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1047/2078] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:39 2022] PUT /identity/v3/projects/1718fd7e002f409cb785964dc96a4465/users/2047b4eacb064fe29f344c96cf7038d2/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:39.884612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.884982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.885329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.912622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:39.914584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] MFA Rules not processed for user `2047b4eacb064fe29f344c96cf7038d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:39.916205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:39.924255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:39.924865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:39.925326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:39.925598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:39.956760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:39.986498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1048/2079] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:39.994276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:39.995935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:39.999670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:40.001097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:40.006173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1032/2080] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:40.075629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21574d-b3ee-4379-86e2-55d1186d384e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:40.079391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:40.080080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:40.080571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:40.080849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:40.090266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:40.091709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:40.101094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1049/2081] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:40.386796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:40.388402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:40.403289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f86b59-908c-401e-8f6a-2f7b355379f3 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:40.406734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:40.407411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:40.407759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:40.408098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:40.410007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:40.472690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:40.472690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:40.475262 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:40.477557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:40.487215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275fd78-ce4d-4bb1-af45-c2949cf9071f service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:40.488665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1033/2082] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:40.505916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:40.506573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:40.506856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:40.507137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] PATH_INFO: `/v3/projects/4dc826493dc049d791cec8b4bb95943f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:40.508497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc826493dc049d791cec8b4bb95943f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:40.510574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:40.652938 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc826493dc049d791cec8b4bb95943f', 'request_id': 'req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:40.728496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc826493dc049d791cec8b4bb95943f', 'request_id': 'req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:40.735734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc826493dc049d791cec8b4bb95943f', 'request_id': 'req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:40.754730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] The token cache is being invalidate because project 4dc826493dc049d791cec8b4bb95943f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:40.756138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:40.756138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:40.757473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1050/2083] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:40 2022] DELETE /identity/v3/projects/4dc826493dc049d791cec8b4bb95943f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:40.761265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:40.772135 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93231eb1-01e6-41d7-ae32-7f79c62232e3 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:40.828677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:40.829302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:40.829583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:40.829891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] PATH_INFO: `/v3/users/7f8371913e3c45ff94ef21b0734d11fa` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:40.831003 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8371913e3c45ff94ef21b0734d11fa)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:40.832939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:41.097867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:41.111924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8371913e3c45ff94ef21b0734d11fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:41.112605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8371913e3c45ff94ef21b0734d11fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:41.120372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8371913e3c45ff94ef21b0734d11fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:41.133546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8371913e3c45ff94ef21b0734d11fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:41.134205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0ace6f7c-32fc-42f9-a78c-3fbec52dca61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8371913e3c45ff94ef21b0734d11fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:41.224116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1034/2084] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:40 2022] DELETE /identity/v3/users/7f8371913e3c45ff94ef21b0734d11fa => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:41.732782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:41.732782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:41.746340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff939c-3a71-4f15-99f0-7b46ca36f7d9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:41.823059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:41.823704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:41.824156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:41.824439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] PATH_INFO: `/v3/projects/a2dfe2e996f44a4694c14d7ffc7a7479` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:41.825575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2dfe2e996f44a4694c14d7ffc7a7479)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:41.827493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:41.829176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:41.940883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2dfe2e996f44a4694c14d7ffc7a7479', 'request_id': 'req-921befda-83aa-4c1d-b936-3a17b3b62be8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:42.017291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2dfe2e996f44a4694c14d7ffc7a7479', 'request_id': 'req-921befda-83aa-4c1d-b936-3a17b3b62be8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:42.024740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2dfe2e996f44a4694c14d7ffc7a7479', 'request_id': 'req-921befda-83aa-4c1d-b936-3a17b3b62be8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:42.035155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] The token cache is being invalidate because project a2dfe2e996f44a4694c14d7ffc7a7479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:42.035452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:42.036092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-921befda-83aa-4c1d-b936-3a17b3b62be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:42.037582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1051/2085] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:41 2022] DELETE /identity/v3/projects/a2dfe2e996f44a4694c14d7ffc7a7479 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:42.042793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d22b6e-3707-4896-a89a-7a622a12b261 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.044468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d22b6e-3707-4896-a89a-7a622a12b261 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.044468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d22b6e-3707-4896-a89a-7a622a12b261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.064821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-78d22b6e-3707-4896-a89a-7a622a12b261 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:42.066089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-78d22b6e-3707-4896-a89a-7a622a12b261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:42.127590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1035/2086] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:42.136181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88146e6-9214-4d97-ab25-2ad52d905d7e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.136725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88146e6-9214-4d97-ab25-2ad52d905d7e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.137071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88146e6-9214-4d97-ab25-2ad52d905d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.160324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a88146e6-9214-4d97-ab25-2ad52d905d7e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:42.161588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a88146e6-9214-4d97-ab25-2ad52d905d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:42.170524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1052/2087] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:42.177806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d22b6e-3707-4896-a89a-7a622a12b261 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:42.187985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d22b6e-3707-4896-a89a-7a622a12b261 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:42.192562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:42.193165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.193437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.193705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.194461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:42.195349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:42.196931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:42.209211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1036/2088] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:42.213836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88146e6-9214-4d97-ab25-2ad52d905d7e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:42.222915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88146e6-9214-4d97-ab25-2ad52d905d7e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:42.227948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:42.228664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.228961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.229247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.229910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:42.231779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:42.358431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1053/2089] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:42 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:42.360219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:42.371387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530dc36-fff4-4e90-8a67-914664ac357e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:42.437389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:42.438022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.438300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.438572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.439202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:42.441692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:42.528705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1037/2090] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:42 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:42.532736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:42.544311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b93ea-3432-4a80-9c8c-09fe3083e4ec admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:42.548952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:42.549548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.549853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.550127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.550764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:42.552372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:42.563495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1054/2091] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:42.567552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:42.577839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29441af2-fbcb-4043-bb64-3bbc6f3a3ec2 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:42.582684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46144b4d-9f6e-4f08-bdfa-633b1bd064bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:42.583319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144b4d-9f6e-4f08-bdfa-633b1bd064bf admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.583597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144b4d-9f6e-4f08-bdfa-633b1bd064bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.583894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144b4d-9f6e-4f08-bdfa-633b1bd064bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.584663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144b4d-9f6e-4f08-bdfa-633b1bd064bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:42.586501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144b4d-9f6e-4f08-bdfa-633b1bd064bf admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:42.597526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1038/2092] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:42.603382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:42.614560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec58d-54ba-44d6-96b6-6ac5639bafba admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:42.620271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:42.620953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.621237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.621526 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] PATH_INFO: `/v3/projects/1fe1a6aebac1439b9fe788df43f1e81e/users/84c70c9e8ccb4058b51c26460bf4ff35/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.637573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe1a6aebac1439b9fe788df43f1e81e, user_id=84c70c9e8ccb4058b51c26460bf4ff35, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:42.640409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:42.733364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1055/2093] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:42 2022] PUT /identity/v3/projects/1fe1a6aebac1439b9fe788df43f1e81e/users/84c70c9e8ccb4058b51c26460bf4ff35/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:42.747008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44febd3-cc30-413a-b0b9-8acbbe1586e7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.747008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44febd3-cc30-413a-b0b9-8acbbe1586e7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.747008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44febd3-cc30-413a-b0b9-8acbbe1586e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.772674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:42.775997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f44febd3-cc30-413a-b0b9-8acbbe1586e7 None None] MFA Rules not processed for user `84c70c9e8ccb4058b51c26460bf4ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:42.777269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f44febd3-cc30-413a-b0b9-8acbbe1586e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:42.845905 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1039/2094] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:42.847016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:42.917242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843f1b2-0f1d-4425-b1ff-a530f031aaa9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:42.922327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:42.922924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:42.923203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:42.923473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:42.924097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:42.926024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:43.042741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1056/2095] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:42 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:43.047658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44febd3-cc30-413a-b0b9-8acbbe1586e7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:43.058499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44febd3-cc30-413a-b0b9-8acbbe1586e7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:43.116942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:43.117763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:43.117946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:43.118700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:43.119373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:43.121305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:43.211487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1040/2096] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:43 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:43.215612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:43.231764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fc3e1-a9fd-4974-bebd-f2dbd356dca8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:43.236668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:43.237276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:43.237561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:43.237866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:43.238537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:43.240322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:43.246781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:43.252629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1057/2097] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:43.258282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:43.268632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c478757-ba33-40f1-b2e0-470fa22b5b17 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:43.274207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:43.274893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:43.275173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:43.275450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] PATH_INFO: `/v3/projects/d8f575f6b451478c91b7fe650e312f55/users/bd109ea7fb4047c48c845605b19a51c5/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:43.290877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f575f6b451478c91b7fe650e312f55, user_id=bd109ea7fb4047c48c845605b19a51c5, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:43.293541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:43.362243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1041/2098] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:43 2022] PUT /identity/v3/projects/d8f575f6b451478c91b7fe650e312f55/users/bd109ea7fb4047c48c845605b19a51c5/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:43.368518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:43.386756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aff65-e50d-4fc3-996a-6d815d3d52c1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:43.450719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0b7df-1483-47d6-8cef-4a147a8198c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:43.451325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0b7df-1483-47d6-8cef-4a147a8198c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:43.451604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0b7df-1483-47d6-8cef-4a147a8198c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:43.451878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0b7df-1483-47d6-8cef-4a147a8198c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:43.452522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0b7df-1483-47d6-8cef-4a147a8198c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:43.454221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0b7df-1483-47d6-8cef-4a147a8198c7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:43.466051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1058/2099] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:43.470393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:43.484857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f611-27d4-421b-9adb-5f4776127ebd admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:43.490695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:43.492470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:43.493015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:43.493391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] PATH_INFO: `/v3/projects/d8f575f6b451478c91b7fe650e312f55/users/bd109ea7fb4047c48c845605b19a51c5/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:43.496876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f575f6b451478c91b7fe650e312f55, user_id=bd109ea7fb4047c48c845605b19a51c5, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:43.499809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:43.512649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1042/2100] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:43 2022] PUT /identity/v3/projects/d8f575f6b451478c91b7fe650e312f55/users/bd109ea7fb4047c48c845605b19a51c5/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:43.522876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0fb6d-1ae1-4cfd-a507-eac9aaa021e7 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:43.523774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0fb6d-1ae1-4cfd-a507-eac9aaa021e7 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:43.524044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0fb6d-1ae1-4cfd-a507-eac9aaa021e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:43.556940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-06e0fb6d-1ae1-4cfd-a507-eac9aaa021e7 None None] MFA Rules not processed for user `bd109ea7fb4047c48c845605b19a51c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:43.558260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-06e0fb6d-1ae1-4cfd-a507-eac9aaa021e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:43.638769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1059/2101] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:43.738767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:43.740582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:43.800011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbad2d7-a10c-438b-b0c5-c3665244d0a9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:43.803581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:43.804193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:43.805022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:43.805401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:43.814477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:43.815924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:43.819903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:43.826727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1043/2102] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:44.070166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0fb6d-1ae1-4cfd-a507-eac9aaa021e7 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:44.079885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0fb6d-1ae1-4cfd-a507-eac9aaa021e7 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:44.084047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fabdd8-dd5c-4eed-945b-1f65b33f6111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:44.084662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fabdd8-dd5c-4eed-945b-1f65b33f6111 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:44.084929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fabdd8-dd5c-4eed-945b-1f65b33f6111 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:44.087465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fabdd8-dd5c-4eed-945b-1f65b33f6111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:44.146194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fabdd8-dd5c-4eed-945b-1f65b33f6111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:44.147922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fabdd8-dd5c-4eed-945b-1f65b33f6111 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:44.157769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1060/2103] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:44.748653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:44.814192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0c7c2-31d5-4c9a-b26a-7a5892538865 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:44.821424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:44.822090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:44.822374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:44.822650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] PATH_INFO: `/v3/users/84c70c9e8ccb4058b51c26460bf4ff35` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:44.823794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c70c9e8ccb4058b51c26460bf4ff35)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:44.825708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:45.073841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c70c9e8ccb4058b51c26460bf4ff35'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.074492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c70c9e8ccb4058b51c26460bf4ff35'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.081632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c70c9e8ccb4058b51c26460bf4ff35'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.094118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c70c9e8ccb4058b51c26460bf4ff35'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.094818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c70c9e8ccb4058b51c26460bf4ff35'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.174347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1044/2104] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:44 2022] DELETE /identity/v3/users/84c70c9e8ccb4058b51c26460bf4ff35 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:45.178908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:45.180015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:45.180165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:45.199504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:45.200874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:45.213620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:45.261442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1061/2105] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:45.271271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:45.273382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:45.282858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:45.284920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:45.300633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e5502-fe7d-482e-a28d-73c33d2c1475 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:45.306017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:45.306644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:45.306925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:45.307202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] PATH_INFO: `/v3/users/6fc3d35d21f84324aeade5e2678f1ed0` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:45.308336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fc3d35d21f84324aeade5e2678f1ed0)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:45.310257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:45.349929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44501706-ba82-4948-bbcd-d6b9f3e2de37 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:45.353822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e481f-3c6b-415a-9d47-b5bf960950db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:45.354441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e481f-3c6b-415a-9d47-b5bf960950db service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:45.354716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e481f-3c6b-415a-9d47-b5bf960950db service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:45.355015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e481f-3c6b-415a-9d47-b5bf960950db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:45.364471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e481f-3c6b-415a-9d47-b5bf960950db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:45.365938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e481f-3c6b-415a-9d47-b5bf960950db service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:45.388916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1045/2106] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:45.570420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fc3d35d21f84324aeade5e2678f1ed0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.571079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fc3d35d21f84324aeade5e2678f1ed0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.577831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fc3d35d21f84324aeade5e2678f1ed0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.594436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fc3d35d21f84324aeade5e2678f1ed0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.595172 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fc3d35d21f84324aeade5e2678f1ed0'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:45.687570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1062/2107] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:45 2022] DELETE /identity/v3/users/6fc3d35d21f84324aeade5e2678f1ed0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:45.692201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:45.692586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:45.693044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:45.712193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:45.713414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:45.725796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:45.780038 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1046/2108] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:45.790798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:45.792213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:45.850265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f1a37-69fc-454f-9add-9d6921422f4a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:45.855002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:45.855604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:45.855884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:45.856159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:45.864944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:45.866417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:45.876343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1063/2109] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:45.929532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:45.930699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:45.944624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0279b1-957c-4702-bcc2-24543f236023 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:45.950366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:45.951207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:45.951664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:45.952215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] PATH_INFO: `/v3/projects/1fe1a6aebac1439b9fe788df43f1e81e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:45.953532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe1a6aebac1439b9fe788df43f1e81e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:45.955497 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:46.122218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe1a6aebac1439b9fe788df43f1e81e', 'request_id': 'req-ee8e9a51-e016-4732-acc4-e99123a2f13b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.217127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe1a6aebac1439b9fe788df43f1e81e', 'request_id': 'req-ee8e9a51-e016-4732-acc4-e99123a2f13b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.222182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe1a6aebac1439b9fe788df43f1e81e', 'request_id': 'req-ee8e9a51-e016-4732-acc4-e99123a2f13b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.234986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] The token cache is being invalidate because project 1fe1a6aebac1439b9fe788df43f1e81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:46.235571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.236809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.239796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1047/2110] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:45 2022] DELETE /identity/v3/projects/1fe1a6aebac1439b9fe788df43f1e81e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:46.243907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:46.254509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fa217-afcf-4763-a4e7-58ec835afc50 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:46.290556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:46.313866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:46.314496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:46.314782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:46.315119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] PATH_INFO: `/v3/users/bd109ea7fb4047c48c845605b19a51c5` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:46.316307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd109ea7fb4047c48c845605b19a51c5)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:46.318262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:46.352684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:46.354258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:46.376667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e9a51-e016-4732-acc4-e99123a2f13b admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:46.381466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:46.382122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:46.385950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:46.385950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] PATH_INFO: `/v3/projects/3b727e42953c4fd195587bb7eeedfb60` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:46.385950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b727e42953c4fd195587bb7eeedfb60)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:46.386345 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:46.494804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd109ea7fb4047c48c845605b19a51c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.496752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd109ea7fb4047c48c845605b19a51c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.498535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b727e42953c4fd195587bb7eeedfb60', 'request_id': 'req-200d556b-092f-4641-b8c8-08de5bd8a8fc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.504913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd109ea7fb4047c48c845605b19a51c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.518333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd109ea7fb4047c48c845605b19a51c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.520995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd109ea7fb4047c48c845605b19a51c5'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.573666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b727e42953c4fd195587bb7eeedfb60', 'request_id': 'req-200d556b-092f-4641-b8c8-08de5bd8a8fc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.580980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b727e42953c4fd195587bb7eeedfb60', 'request_id': 'req-200d556b-092f-4641-b8c8-08de5bd8a8fc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.591852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] The token cache is being invalidate because project 3b727e42953c4fd195587bb7eeedfb60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:46.592140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.592804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:46.594335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1048/2111] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:46 2022] DELETE /identity/v3/projects/3b727e42953c4fd195587bb7eeedfb60 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:46.598211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:46.599587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:46.655733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1064/2112] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:46 2022] DELETE /identity/v3/users/bd109ea7fb4047c48c845605b19a51c5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:46.674506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d556b-092f-4641-b8c8-08de5bd8a8fc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:46.730896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:46.731442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:46.731722 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:46.731997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] PATH_INFO: `/v3/users/51ae6e6ebe164c109150da1a8ca0b0dc` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:46.733297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ae6e6ebe164c109150da1a8ca0b0dc)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:46.735353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:46.974368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:47.111194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ae6e6ebe164c109150da1a8ca0b0dc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.111882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ae6e6ebe164c109150da1a8ca0b0dc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.119491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ae6e6ebe164c109150da1a8ca0b0dc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.131272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ae6e6ebe164c109150da1a8ca0b0dc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.131950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-17f2f84c-1143-4c4d-b9c7-1b0436d05101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ae6e6ebe164c109150da1a8ca0b0dc'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.198507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:47.199984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:47.215990 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8624835-6e27-4481-a880-d35461c24fd9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:47.220795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:47.221425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:47.221703 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:47.222014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] PATH_INFO: `/v3/projects/d8f575f6b451478c91b7fe650e312f55` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:47.223148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f575f6b451478c91b7fe650e312f55)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:47.225068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:47.245699 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1049/2113] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:46 2022] DELETE /identity/v3/users/51ae6e6ebe164c109150da1a8ca0b0dc => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:47.376985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f575f6b451478c91b7fe650e312f55', 'request_id': 'req-0f582988-0063-4a27-88a0-dcf36b735dfa'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.442017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f575f6b451478c91b7fe650e312f55', 'request_id': 'req-0f582988-0063-4a27-88a0-dcf36b735dfa'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.449091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f575f6b451478c91b7fe650e312f55', 'request_id': 'req-0f582988-0063-4a27-88a0-dcf36b735dfa'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.460554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] The token cache is being invalidate because project d8f575f6b451478c91b7fe650e312f55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:47.460963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.462426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f582988-0063-4a27-88a0-dcf36b735dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.465162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1065/2114] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:47 2022] DELETE /identity/v3/projects/d8f575f6b451478c91b7fe650e312f55 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:47.470492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:47.470773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:47.471045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:47.517256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:47.518583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:47.612174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1050/2115] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:47.623795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25499-871c-465f-8508-5cf0ada191dd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:47.624139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25499-871c-465f-8508-5cf0ada191dd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:47.624683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25499-871c-465f-8508-5cf0ada191dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:47.646906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:47.648355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:47.660578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940aa26-3264-4e00-a1ef-5bb44943950f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:47.666007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-57b25499-871c-465f-8508-5cf0ada191dd None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:47.666340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:47.667116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:47.667393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:47.667549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-57b25499-871c-465f-8508-5cf0ada191dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:47.668584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] PATH_INFO: `/v3/users/b909f8a5a75c482c81d493e029b99095` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:47.669602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=b909f8a5a75c482c81d493e029b99095)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:47.671621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:47.676982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1066/2116] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:47.681547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25499-871c-465f-8508-5cf0ada191dd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:47.691548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25499-871c-465f-8508-5cf0ada191dd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:47.697225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:47.697862 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:47.702209 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:47.703131 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:47.703503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:47.703888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:47.705614 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:47.721774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1067/2117] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:06:47.725944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:47.740941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f2ee8-1514-4d71-9d72-cc9aa87fcb38 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:47.746593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:47.747272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:47.747616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:47.747949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:47.748680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:47.750700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:47.767341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:47.876891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1068/2118] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:47 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:47.876891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:47.879922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:47.881356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:47.896212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b909f8a5a75c482c81d493e029b99095'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.896894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b909f8a5a75c482c81d493e029b99095'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.905652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b909f8a5a75c482c81d493e029b99095'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.929579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b909f8a5a75c482c81d493e029b99095'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.930333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b909f8a5a75c482c81d493e029b99095'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:47.948643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033cd06-fd9e-4e3d-b7b0-9d2b128daa1e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:47.953384 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:47.954042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:47.954323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:47.954598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] PATH_INFO: `/v3/projects/a8f6c3f2754d423faa83c17d132689e3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:47.955825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f6c3f2754d423faa83c17d132689e3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:47.957777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.074342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1051/2119] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:47 2022] DELETE /identity/v3/users/b909f8a5a75c482c81d493e029b99095 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:48.075246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.076619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:48.139337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cf623-39da-4efe-9992-2fb576e3c6de admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.143879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a0d5f-5c0a-473f-bbd2-21c6608851e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.144470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a0d5f-5c0a-473f-bbd2-21c6608851e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.144744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a0d5f-5c0a-473f-bbd2-21c6608851e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.145011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a0d5f-5c0a-473f-bbd2-21c6608851e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.145636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a0d5f-5c0a-473f-bbd2-21c6608851e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.147686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a0d5f-5c0a-473f-bbd2-21c6608851e3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.167715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f6c3f2754d423faa83c17d132689e3', 'request_id': 'req-9591c379-49f6-499a-a7ba-4e5e9fc6f316'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:48.291977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1052/2120] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:48.293436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abab04-706e-4d6b-a64b-b283e2c45ad4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.293723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abab04-706e-4d6b-a64b-b283e2c45ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.294276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abab04-706e-4d6b-a64b-b283e2c45ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.294453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f6c3f2754d423faa83c17d132689e3', 'request_id': 'req-9591c379-49f6-499a-a7ba-4e5e9fc6f316'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:48.302067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f6c3f2754d423faa83c17d132689e3', 'request_id': 'req-9591c379-49f6-499a-a7ba-4e5e9fc6f316'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:48.313020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-61abab04-706e-4d6b-a64b-b283e2c45ad4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:48.313245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] The token cache is being invalidate because project a8f6c3f2754d423faa83c17d132689e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:48.313535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:48.314232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:48.314335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-61abab04-706e-4d6b-a64b-b283e2c45ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:48.315764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1069/2121] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:47 2022] DELETE /identity/v3/projects/a8f6c3f2754d423faa83c17d132689e3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:48.316587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.317840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:48.375207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1053/2122] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:48.376730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.377011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.377283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.386240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591c379-49f6-499a-a7ba-4e5e9fc6f316 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.391097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.391740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.392025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.392302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.393009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.394643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.397308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:48.398535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:48.405865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1070/2123] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:48.407396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1054/2124] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:48.408245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.409439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:48.409871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.421300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ea056-1ec8-4826-bac4-429730bfdfe8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.426040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a01871-c0ec-4b21-b0a1-1ffa36387bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.426704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a01871-c0ec-4b21-b0a1-1ffa36387bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.426991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a01871-c0ec-4b21-b0a1-1ffa36387bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.427317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a01871-c0ec-4b21-b0a1-1ffa36387bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.427966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a01871-c0ec-4b21-b0a1-1ffa36387bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.429547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a01871-c0ec-4b21-b0a1-1ffa36387bbf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.440202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1071/2125] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:48.441453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685381-3531-45b4-b965-4287d4428b24 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.441768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685381-3531-45b4-b965-4287d4428b24 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.442048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685381-3531-45b4-b965-4287d4428b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.463056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-de685381-3531-45b4-b965-4287d4428b24 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:48.464253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-de685381-3531-45b4-b965-4287d4428b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:48.470115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257fda7-d1f1-442d-aa0b-831459998534 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.472206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1072/2126] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:48.473100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de685381-3531-45b4-b965-4287d4428b24 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.477007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.478536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.478754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.479941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.482724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de685381-3531-45b4-b965-4287d4428b24 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.487390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faebe0-29fc-41be-8635-443ef7bdd892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.488047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faebe0-29fc-41be-8635-443ef7bdd892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.488329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faebe0-29fc-41be-8635-443ef7bdd892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.488599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faebe0-29fc-41be-8635-443ef7bdd892 admin admin] PATH_INFO: `/v3/projects/189f2c9a669a46f68369bb3c6b058f8f/users/2c58d4800ba54bc58779466339406fab/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.490117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.491551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.500197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:48.501371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1055/2127] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:48.502331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.506312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faebe0-29fc-41be-8635-443ef7bdd892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189f2c9a669a46f68369bb3c6b058f8f, user_id=2c58d4800ba54bc58779466339406fab, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.509766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faebe0-29fc-41be-8635-443ef7bdd892 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.512486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d9c64-5dc3-4f28-b03b-b3947c29e692 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.517223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.517786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.518071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.518360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.519001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:48.520158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.521299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.532456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:48.546541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1056/2128] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:48.548095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.558629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868f4d7-1bb4-4e62-8147-75453ff3223f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.563438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.564039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.569612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.569612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.569612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.569612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.597460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1073/2129] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:48 2022] PUT /identity/v3/projects/189f2c9a669a46f68369bb3c6b058f8f/users/2c58d4800ba54bc58779466339406fab/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:48.610061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dae55-7f70-4c80-bd53-f3c091eb9f84 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.610355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dae55-7f70-4c80-bd53-f3c091eb9f84 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.610636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dae55-7f70-4c80-bd53-f3c091eb9f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.640259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-720dae55-7f70-4c80-bd53-f3c091eb9f84 None None] MFA Rules not processed for user `2c58d4800ba54bc58779466339406fab`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:48.641532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-720dae55-7f70-4c80-bd53-f3c091eb9f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:48.706228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1074/2130] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:48.710614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1057/2131] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:48.710953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dae55-7f70-4c80-bd53-f3c091eb9f84 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.711479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.787909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dae55-7f70-4c80-bd53-f3c091eb9f84 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.789772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.790550 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.791116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.791460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.792031 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fab41-3679-4da8-a89d-cb253b3f620c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.792229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.794632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.798218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.798827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.799105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.799376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.800102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.802228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.886132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1075/2132] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:48.890365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.907539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1058/2133] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:48.911970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:48.923911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78742c-6fb7-4770-b578-203417329a39 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.965180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1e86a-cbf4-4e9e-9894-ad8e0a221234 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:48.970930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.971570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.973060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.973060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.973060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.974809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.984011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:48.984640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:48.984969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:48.985656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:48.985948 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:48.987667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:48.999115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1059/2134] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:49.003178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.013708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dda626-6e4c-4553-bc44-9b43f338370c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.019075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.019783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.020088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.020357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.020996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.022614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.034464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1060/2135] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:49.035147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.036835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.051693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fc09c-7170-43c9-8689-d937623927df admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.060813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.060813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.060813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.060813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] PATH_INFO: `/v3/projects/4fd30b71c7ee4e859079e7282a84bf98` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.061480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd30b71c7ee4e859079e7282a84bf98)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.062155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.101798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1076/2136] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:48 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:49.101798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.114554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa4c83-0270-4589-a532-8858afb3877e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.120458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.121136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.121410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.121678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] PATH_INFO: `/v3/projects/3c253b4458d247828e40a3581b46dbef/users/6fd08627fe0c4f44b6b2797693231c3e/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.135241 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:49.138455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c253b4458d247828e40a3581b46dbef, user_id=6fd08627fe0c4f44b6b2797693231c3e, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.141102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.251550 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1077/2137] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:49 2022] PUT /identity/v3/projects/3c253b4458d247828e40a3581b46dbef/users/6fd08627fe0c4f44b6b2797693231c3e/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:49.260653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.260784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:49.267703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd30b71c7ee4e859079e7282a84bf98', 'request_id': 'req-5f2613ec-91bd-4404-9c3a-a4ca955d2999'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.314799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17398ed5-6096-4c1d-9ec6-cf0c29d3f8bc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.339711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd30b71c7ee4e859079e7282a84bf98', 'request_id': 'req-5f2613ec-91bd-4404-9c3a-a4ca955d2999'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.346480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd30b71c7ee4e859079e7282a84bf98', 'request_id': 'req-5f2613ec-91bd-4404-9c3a-a4ca955d2999'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.357598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] The token cache is being invalidate because project 4fd30b71c7ee4e859079e7282a84bf98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:49.357929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.361823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5f2613ec-91bd-4404-9c3a-a4ca955d2999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.361823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1061/2138] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:49 2022] DELETE /identity/v3/projects/4fd30b71c7ee4e859079e7282a84bf98 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:49.361823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.361823 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.362115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.379707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.380314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.380591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.380865 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.381508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.383115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.391529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] MFA Rules not processed for user `6fd08627fe0c4f44b6b2797693231c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:49.392724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.392998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1078/2139] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:49.393844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.397289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.409622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633e1c5-be2a-4c38-88c6-291c67037f68 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.414282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.419296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.419296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.419296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] PATH_INFO: `/v3/users/2047b4eacb064fe29f344c96cf7038d2` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.419296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2047b4eacb064fe29f344c96cf7038d2)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.419296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.463590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1062/2140] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:49.464581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.465914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.479323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f965a-df93-4473-9eaa-389d975715b4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.484156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.484758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.485036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.485310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.486010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.487600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.498571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1063/2141] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:49.499344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.500935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.513036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030ca41-0a7f-43ed-85bf-2ed3c3c378ca admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.518078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.518657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.518937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.519244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.519870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.521815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.612885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1064/2142] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:49 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:49.613887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.625575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:49.659341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2047b4eacb064fe29f344c96cf7038d2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.659951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2047b4eacb064fe29f344c96cf7038d2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.669449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2047b4eacb064fe29f344c96cf7038d2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.671615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:49.683662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2047b4eacb064fe29f344c96cf7038d2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.684366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-55054a49-9728-4879-a787-f4bba29dd66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2047b4eacb064fe29f344c96cf7038d2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:49.686470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a91af-d272-4275-98ea-3b8f643438eb admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.691365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.692030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.692313 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.692622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] PATH_INFO: `/v3/projects/b7d1cbe564354e6e89a2747fe4121e4b/users/24a2862a21644384bfec1539eeb651b6/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.708828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d1cbe564354e6e89a2747fe4121e4b, user_id=24a2862a21644384bfec1539eeb651b6, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.711622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.789037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1065/2144] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:49 2022] PUT /identity/v3/projects/b7d1cbe564354e6e89a2747fe4121e4b/users/24a2862a21644384bfec1539eeb651b6/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:49.789239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1079/2144] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:49 2022] DELETE /identity/v3/users/2047b4eacb064fe29f344c96cf7038d2 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:49.789987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.792042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.799510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.800298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.800648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.833299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] MFA Rules not processed for user `24a2862a21644384bfec1539eeb651b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:49.834672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.873390 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9200d-b6e5-480c-93da-7c7c093131d0 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.910471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1080/2145] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:49.915446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:49.916978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:49.936639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.937249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.937525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.937956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.938467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.940366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:49.984486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729aab7-a763-4e63-9c55-8dfe6f316f3c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:49.989145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:49.989781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:49.990065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:49.990337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:49.990997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:49.994106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.083004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1066/2146] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:49 2022] POST /identity/v3/users => generated 297 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:50.088832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.097365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b106e00-95cd-4a5d-ab2a-7b6a0ab835b6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.103160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.104023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.104310 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.104588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.105233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.107093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.117528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1067/2147] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:50.121474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1081/2148] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:49 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:50.124735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.124870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.136389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c984b-999d-4e6e-8ea5-c90e3f7f488a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.159137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:50.218944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:50.232781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.234133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.235129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.235442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] PATH_INFO: `/v3/projects/ae2527b5bbab4ee2a63ac74cce55bab4/users/d3a4e8c49d914228a7ea0c11faa06fda/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.240116 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad00fe-e24b-4ee5-ab10-cc87da29e57c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.246936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.247567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.247854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.248128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.248772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.251558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.258056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2527b5bbab4ee2a63ac74cce55bab4, user_id=d3a4e8c49d914228a7ea0c11faa06fda, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.261714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.349787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1068/2149] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:50 2022] PUT /identity/v3/projects/ae2527b5bbab4ee2a63ac74cce55bab4/users/d3a4e8c49d914228a7ea0c11faa06fda/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:50.353449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.355287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:50.384523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1082/2150] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:50 2022] POST /identity/v3/users => generated 295 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:50.386345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.401149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce043e1-cb2b-4bc6-bd9e-29999c4730c5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.448548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa2210-2ddb-40c0-be7e-c78bd07373cc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.456519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.457421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.457697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.458272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] PATH_INFO: `/v3/projects/1718fd7e002f409cb785964dc96a4465` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.459430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1718fd7e002f409cb785964dc96a4465)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.461352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.469607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.470554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.470909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.471198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.472012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.473694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.485839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1083/2151] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:50.487106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.500765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb3acd-306d-40ff-b0c2-16c9a2e0244a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.506715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.507872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.507872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.508236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.509536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.511797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.523429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1084/2152] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:50.532100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.541487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebcaad-8129-44a2-abb1-59cc0e258edd admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.547149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.547883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.548226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.548502 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] PATH_INFO: `/v3/projects/ae2527b5bbab4ee2a63ac74cce55bab4/users/d3a4e8c49d914228a7ea0c11faa06fda/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.550679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2527b5bbab4ee2a63ac74cce55bab4, user_id=d3a4e8c49d914228a7ea0c11faa06fda, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.556225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.564695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1085/2153] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:50 2022] PUT /identity/v3/projects/ae2527b5bbab4ee2a63ac74cce55bab4/users/d3a4e8c49d914228a7ea0c11faa06fda/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:50.565482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.622164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1718fd7e002f409cb785964dc96a4465', 'request_id': 'req-994db679-7f7a-467b-aec2-5654e61f1b22'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:50.640244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cf9a1-20eb-4882-9681-517e5e1dca41 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.697817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1718fd7e002f409cb785964dc96a4465', 'request_id': 'req-994db679-7f7a-467b-aec2-5654e61f1b22'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:50.705628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.705628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.705628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.705628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] PATH_INFO: `/v3/projects/a94599aa400a463c95706fefaeb4f70f/users/1b09ed7e0a5549678994a9e5aa72945a/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.706160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1718fd7e002f409cb785964dc96a4465', 'request_id': 'req-994db679-7f7a-467b-aec2-5654e61f1b22'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:50.718745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] The token cache is being invalidate because project 1718fd7e002f409cb785964dc96a4465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:50.719129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:50.720070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-994db679-7f7a-467b-aec2-5654e61f1b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:50.721913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94599aa400a463c95706fefaeb4f70f, user_id=1b09ed7e0a5549678994a9e5aa72945a, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.722151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1069/2154] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:50 2022] DELETE /identity/v3/projects/1718fd7e002f409cb785964dc96a4465 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:50.723671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bfa8e-23ac-4ecf-8bcb-ded3a5f8efbb None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.723946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bfa8e-23ac-4ecf-8bcb-ded3a5f8efbb None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.724224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2bfa8e-23ac-4ecf-8bcb-ded3a5f8efbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.729915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:50.730459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.758654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7a2bfa8e-23ac-4ecf-8bcb-ded3a5f8efbb None None] MFA Rules not processed for user `d3a4e8c49d914228a7ea0c11faa06fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:50.760066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7a2bfa8e-23ac-4ecf-8bcb-ded3a5f8efbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:50.782595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:50.834997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1070/2155] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:50.836533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ee6d2-78e0-4fa1-8d7a-6118608fc985 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.836849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ee6d2-78e0-4fa1-8d7a-6118608fc985 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.837120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ee6d2-78e0-4fa1-8d7a-6118608fc985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.856454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1086/2156] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:50 2022] PUT /identity/v3/projects/a94599aa400a463c95706fefaeb4f70f/users/1b09ed7e0a5549678994a9e5aa72945a/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:50.858834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-3f9ee6d2-78e0-4fa1-8d7a-6118608fc985 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:50.860078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3f9ee6d2-78e0-4fa1-8d7a-6118608fc985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:50.860277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.861874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:50.921056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1071/2157] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:50.928184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f0aeb-2f89-42c4-9c94-063ce709b74d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.928820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266f74-b680-4e36-a7d6-55dc663f2b7c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.929104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266f74-b680-4e36-a7d6-55dc663f2b7c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.929382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266f74-b680-4e36-a7d6-55dc663f2b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.933873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.934478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.934765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.935041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.935682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.937317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.949295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1087/2158] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:50.951992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-72266f74-b680-4e36-a7d6-55dc663f2b7c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:50.952457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.953228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-72266f74-b680-4e36-a7d6-55dc663f2b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:50.961223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1072/2159] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:50.962719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f025e-1e1a-48a3-a006-b74d47182064 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.966672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266f74-b680-4e36-a7d6-55dc663f2b7c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:50.971161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2aa51-03bb-4f1d-a0f9-97459ae67801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.972262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2aa51-03bb-4f1d-a0f9-97459ae67801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.973942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2aa51-03bb-4f1d-a0f9-97459ae67801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.974839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2aa51-03bb-4f1d-a0f9-97459ae67801 admin admin] PATH_INFO: `/v3/projects/a94599aa400a463c95706fefaeb4f70f/users/1b09ed7e0a5549678994a9e5aa72945a/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.976999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266f74-b680-4e36-a7d6-55dc663f2b7c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:50.977798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2aa51-03bb-4f1d-a0f9-97459ae67801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94599aa400a463c95706fefaeb4f70f, user_id=1b09ed7e0a5549678994a9e5aa72945a, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:50.981196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2aa51-03bb-4f1d-a0f9-97459ae67801 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:50.995762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:50.997820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:50.997820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:50.997820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:50.998196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1088/2160] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:50 2022] PUT /identity/v3/projects/a94599aa400a463c95706fefaeb4f70f/users/1b09ed7e0a5549678994a9e5aa72945a/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:06:51.001569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:51.001569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.001569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.012239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.012522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.012797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.021876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1073/2161] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:51.023376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.036949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b8fce-2723-490c-8526-e97e2d9faa8d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.063444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] MFA Rules not processed for user `1b09ed7e0a5549678994a9e5aa72945a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:51.064689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:51.110287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.111352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.111895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.112370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.113222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.115585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.155720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1089/2162] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:51.156844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.158805 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:51.219806 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40c81e-d107-481e-8fb0-e7b0da4063d4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.223366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.223967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.224296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.224567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.229302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:51.283157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.284601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.297777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1090/2163] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:51.301771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1074/2164] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:51 2022] POST /identity/v3/projects => generated 373 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:51.305389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.316413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844e806-d3b7-4dde-bbc5-e3e1a4fa2a79 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.384056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.384394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.384675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.384958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.385233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.385893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.387950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.415161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:51.442308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046de8-f8c2-4789-ac7d-5f119e5b5605 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.446499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.447247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.447531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.447809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.449553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:51.486659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1091/2165] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:51 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:51.490543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.500371 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315899e-ef3a-43c9-84fe-d3391c9dab78 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.505033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.505626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.505939 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.506216 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.506863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.508449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.519358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1092/2166] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:51.519628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.521070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.523383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.531131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1075/2167] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:51.534543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6feb8c-cc7d-4a51-bcbf-30515188c685 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.539318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f6351-b087-470b-ad82-7f5ab77265a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.540068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f6351-b087-470b-ad82-7f5ab77265a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.540344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f6351-b087-470b-ad82-7f5ab77265a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.540666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f6351-b087-470b-ad82-7f5ab77265a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.541297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f6351-b087-470b-ad82-7f5ab77265a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.542932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f6351-b087-470b-ad82-7f5ab77265a0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.555221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1093/2168] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:06:51.559626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.572471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98114f66-2671-4a57-8f3d-02869f24a200 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.577042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.577691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.578109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.578388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] PATH_INFO: `/v3/projects/9788b5b66a0c452cb39b263ca30284e0/users/4690104f5bea480d87b7c43488182441/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.594587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9788b5b66a0c452cb39b263ca30284e0, user_id=4690104f5bea480d87b7c43488182441, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.597250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.672177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1076/2169] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:51 2022] PUT /identity/v3/projects/9788b5b66a0c452cb39b263ca30284e0/users/4690104f5bea480d87b7c43488182441/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:51.677243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4c8a0-6042-43eb-94fd-8e1273701dd6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.677549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4c8a0-6042-43eb-94fd-8e1273701dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.677860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4c8a0-6042-43eb-94fd-8e1273701dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.696970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ddc4c8a0-6042-43eb-94fd-8e1273701dd6 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:51.698223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ddc4c8a0-6042-43eb-94fd-8e1273701dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:51.766284 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1094/2170] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:51.776571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:51.778374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:51.845281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206eec-5101-4706-8268-0727dfaac02e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:51.849061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:51.849766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:51.850107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:51.850466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:51.862164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:51.866024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:51.875063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1077/2171] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:53.000995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4c8a0-6042-43eb-94fd-8e1273701dd6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:53.011112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:53.063797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4c8a0-6042-43eb-94fd-8e1273701dd6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:53.068267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3a4e8c49d914228a7ea0c11faa06fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae2527b5bbab4ee2a63ac74cce55bab4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:53.068872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:53.069157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:53.069434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:53.070105 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:53.070414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:53.072024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:53.084483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1095/2172] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:53.088582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:53.099117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035f821-dc44-42fd-b25b-feea11314c31 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:53.104223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3a4e8c49d914228a7ea0c11faa06fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae2527b5bbab4ee2a63ac74cce55bab4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:53.104834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:53.105119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:53.105398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:53.106147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:53.108041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:53.246697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1078/2173] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:53 2022] POST /identity/v3/projects => generated 355 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:53.250776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:53.263080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ceedb-ba3e-4468-8f1c-24879ba7ada4 tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:53.337445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3a4e8c49d914228a7ea0c11faa06fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae2527b5bbab4ee2a63ac74cce55bab4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:53.337445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:53.337445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:53.338484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:53.338484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:53.338804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:53.340759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:53.359299 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1096/2174] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:06:53.398597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:53.411404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a06f0-49bf-4261-9cc1-3b79ac54453b tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:53.417516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3a4e8c49d914228a7ea0c11faa06fda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae2527b5bbab4ee2a63ac74cce55bab4', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:53.418208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:53.418558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:53.419094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] PATH_INFO: `/v3/projects/30d1d895307048a588534d6b4b2ac4e0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:53.420442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorizing `identity:delete_project(project_id=30d1d895307048a588534d6b4b2ac4e0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:53.422940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:53.557304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:53.582460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30d1d895307048a588534d6b4b2ac4e0', 'request_id': 'req-59be28b7-208c-412f-99ea-07d38d3edaca'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:53.610641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:53.675069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30d1d895307048a588534d6b4b2ac4e0', 'request_id': 'req-59be28b7-208c-412f-99ea-07d38d3edaca'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:53.691371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30d1d895307048a588534d6b4b2ac4e0', 'request_id': 'req-59be28b7-208c-412f-99ea-07d38d3edaca'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:53.705455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] The token cache is being invalidate because project 30d1d895307048a588534d6b4b2ac4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:53.705961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:53.706857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:53.708705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1079/2175] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:53 2022] DELETE /identity/v3/projects/30d1d895307048a588534d6b4b2ac4e0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:53.738303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be3d90-a58d-4f77-8260-7b7057a5f0ff tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:53.852668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e19917-f48e-4fae-9085-1ef13b65e538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:53.853887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19917-f48e-4fae-9085-1ef13b65e538 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:53.854358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19917-f48e-4fae-9085-1ef13b65e538 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:53.854801 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e19917-f48e-4fae-9085-1ef13b65e538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:53.858873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-73e19917-f48e-4fae-9085-1ef13b65e538 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:53.870425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:53.948029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e19917-f48e-4fae-9085-1ef13b65e538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:53.949478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e19917-f48e-4fae-9085-1ef13b65e538 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:53.960193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1097/2176] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:06:54.545918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:54.547406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:54.646581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be28b7-208c-412f-99ea-07d38d3edaca tempest-VolumeQuotaClassesTest-1895407563 tempest-VolumeQuotaClassesTest-1895407563-project-admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:54.652230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:54.653478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:54.653782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:54.654062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] PATH_INFO: `/v3/users/6fd08627fe0c4f44b6b2797693231c3e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:54.655206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd08627fe0c4f44b6b2797693231c3e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:54.657226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:54.909930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd08627fe0c4f44b6b2797693231c3e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:54.910822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd08627fe0c4f44b6b2797693231c3e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:54.920082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd08627fe0c4f44b6b2797693231c3e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:54.940207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd08627fe0c4f44b6b2797693231c3e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:54.941815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd08627fe0c4f44b6b2797693231c3e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:55.093383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1080/2177] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:54 2022] DELETE /identity/v3/users/6fd08627fe0c4f44b6b2797693231c3e => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:55.097670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:55.098158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:55.098332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:55.130648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:55.136452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:55.208269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1098/2178] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:55.219654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:55.222342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:55.305292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52122-cfab-41ad-a7f9-30899df38e2c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:55.312454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:55.313557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:55.314088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:55.314564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:55.330048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:55.331862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:55.347240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1081/2179] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:55.390350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:55.829043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:55.831803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:55.906190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0bcbff-06b9-453a-8455-b9531a60a6fb None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:55.911038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba5155-e2c3-4580-9bab-8fd3a7fefbc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:55.911386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba5155-e2c3-4580-9bab-8fd3a7fefbc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:55.911672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba5155-e2c3-4580-9bab-8fd3a7fefbc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:55.911985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba5155-e2c3-4580-9bab-8fd3a7fefbc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:55.913795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c1ba5155-e2c3-4580-9bab-8fd3a7fefbc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:55.981901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba5155-e2c3-4580-9bab-8fd3a7fefbc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:55.983345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba5155-e2c3-4580-9bab-8fd3a7fefbc5 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:55.989643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:55.995182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1099/2180] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:56.019792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:56.021259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:56.037610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cd2e6-f945-49b5-8ce7-7afb708d6dd3 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:56.049881 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:56.050529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:56.050814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:56.051086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] PATH_INFO: `/v3/projects/3c253b4458d247828e40a3581b46dbef` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:56.052696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c253b4458d247828e40a3581b46dbef)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:56.054808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:56.083585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc360a1-6682-43fb-aa32-0b787b54d6bf None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:56.083871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc360a1-6682-43fb-aa32-0b787b54d6bf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:56.084367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc360a1-6682-43fb-aa32-0b787b54d6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:56.105355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fcc360a1-6682-43fb-aa32-0b787b54d6bf None None] MFA Rules not processed for user `9fadb80f21614816b3526d59b2d8a0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:56.107001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fcc360a1-6682-43fb-aa32-0b787b54d6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:56.201055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1100/2181] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:56.226713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c253b4458d247828e40a3581b46dbef', 'request_id': 'req-a730446d-9709-4455-8164-5cefce1a6268'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.299672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c253b4458d247828e40a3581b46dbef', 'request_id': 'req-a730446d-9709-4455-8164-5cefce1a6268'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.306922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c253b4458d247828e40a3581b46dbef', 'request_id': 'req-a730446d-9709-4455-8164-5cefce1a6268'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.319845 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] The token cache is being invalidate because project 3c253b4458d247828e40a3581b46dbef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:56.320417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.321293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.323099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1082/2182] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:56 2022] DELETE /identity/v3/projects/3c253b4458d247828e40a3581b46dbef => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:56.327663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc360a1-6682-43fb-aa32-0b787b54d6bf None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:56.342483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc360a1-6682-43fb-aa32-0b787b54d6bf None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:56.447579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:56.448712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:56.449178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:56.449428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] PATH_INFO: `/v3/users/d3a4e8c49d914228a7ea0c11faa06fda` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:56.450698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a4e8c49d914228a7ea0c11faa06fda)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:56.452777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:56.699357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a4e8c49d914228a7ea0c11faa06fda'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.700403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a4e8c49d914228a7ea0c11faa06fda'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.708342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a4e8c49d914228a7ea0c11faa06fda'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.728232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a4e8c49d914228a7ea0c11faa06fda'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.733447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70026ab6-1b4c-46b8-a444-c0a8c61d4e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a4e8c49d914228a7ea0c11faa06fda'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:56.884853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1101/2183] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:06:56 2022] DELETE /identity/v3/users/d3a4e8c49d914228a7ea0c11faa06fda => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:57.698941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:57.700745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:57.728017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730446d-9709-4455-8164-5cefce1a6268 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:57.889529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:57.890482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:57.891047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:57.891592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] PATH_INFO: `/v3/projects/ae2527b5bbab4ee2a63ac74cce55bab4` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:57.893368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2527b5bbab4ee2a63ac74cce55bab4)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:57.896491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:58.037620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2527b5bbab4ee2a63ac74cce55bab4', 'request_id': 'req-fd650e3c-b73e-4e99-9da2-0c66b3249696'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:58.128360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2527b5bbab4ee2a63ac74cce55bab4', 'request_id': 'req-fd650e3c-b73e-4e99-9da2-0c66b3249696'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:58.137920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2527b5bbab4ee2a63ac74cce55bab4', 'request_id': 'req-fd650e3c-b73e-4e99-9da2-0c66b3249696'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:58.144533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:58.160806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] The token cache is being invalidate because project ae2527b5bbab4ee2a63ac74cce55bab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:06:58.163169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:58.165989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fd650e3c-b73e-4e99-9da2-0c66b3249696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:06:58.167550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1083/2184] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:06:57 2022] DELETE /identity/v3/projects/ae2527b5bbab4ee2a63ac74cce55bab4 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:58.172932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0f9c2-51a2-40a9-8506-d5981c873554 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.173429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0f9c2-51a2-40a9-8506-d5981c873554 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.173634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0f9c2-51a2-40a9-8506-d5981c873554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:58.215430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-96a0f9c2-51a2-40a9-8506-d5981c873554 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:58.217083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-96a0f9c2-51a2-40a9-8506-d5981c873554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:58.248021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:06:58.317255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1102/2185] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:58.325399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f736-3d7a-4546-96b7-05cc9d1f7034 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.325684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f736-3d7a-4546-96b7-05cc9d1f7034 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.325994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f736-3d7a-4546-96b7-05cc9d1f7034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:58.347793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7441f736-3d7a-4546-96b7-05cc9d1f7034 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:58.349157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7441f736-3d7a-4546-96b7-05cc9d1f7034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:58.358241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1084/2186] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:58.362628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0f9c2-51a2-40a9-8506-d5981c873554 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:58.373842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0f9c2-51a2-40a9-8506-d5981c873554 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:58.378844 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:58.379489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.379772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.380049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:58.380846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:06:58.381026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:58.382700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:58.395107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1103/2187] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:06:58.399636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441f736-3d7a-4546-96b7-05cc9d1f7034 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:58.420609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441f736-3d7a-4546-96b7-05cc9d1f7034 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:58.432070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:58.432862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.433124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.433403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:58.440493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:58.442864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:58.576919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1085/2188] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:58 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:58.581285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:58.600122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340281b-389d-4fcc-8772-59acd00fd65d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:58.688114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:58.688851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.689193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.689530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:58.690377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:58.692837 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:58.794554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1104/2189] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:58 2022] POST /identity/v3/users => generated 284 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:06:58.799182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:58.814182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53dbd3-6f3a-4baf-b1ec-9a2fb7c3fcc3 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:58.823802 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:58.824546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.825000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.825434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:58.826687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:58.829573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:58.846889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1086/2190] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:58.850735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:58.878704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371008-1d80-4d1e-a04c-588f1204eeb6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:58.897480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26596a50-1b30-4f11-b672-5bde8af7c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:58.901467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596a50-1b30-4f11-b672-5bde8af7c740 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.902956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596a50-1b30-4f11-b672-5bde8af7c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.903419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596a50-1b30-4f11-b672-5bde8af7c740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:58.906032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26596a50-1b30-4f11-b672-5bde8af7c740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:58.913391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26596a50-1b30-4f11-b672-5bde8af7c740 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:58.963182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:58.964503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1105/2191] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:58.983328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9568779-b3ff-4fbf-8d1d-592004b18e79 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:58.990188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8a4d1-c231-4da0-b484-caf9631c4957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:58.992164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8a4d1-c231-4da0-b484-caf9631c4957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:58.992492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8a4d1-c231-4da0-b484-caf9631c4957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:58.992844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8a4d1-c231-4da0-b484-caf9631c4957 admin admin] PATH_INFO: `/v3/projects/743eb3a5068f4b59bc85e7fb17c83787/users/ad0978e6967348668bf9f0b45f252c5c/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:59.018800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8a4d1-c231-4da0-b484-caf9631c4957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743eb3a5068f4b59bc85e7fb17c83787, user_id=ad0978e6967348668bf9f0b45f252c5c, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:59.021873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8a4d1-c231-4da0-b484-caf9631c4957 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:59.174534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1087/2192] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:58 2022] PUT /identity/v3/projects/743eb3a5068f4b59bc85e7fb17c83787/users/ad0978e6967348668bf9f0b45f252c5c/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:06:59.181839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:59.182355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:59.182527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:59.198046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:59.198936 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:59.199893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:59.269909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] MFA Rules not processed for user `ad0978e6967348668bf9f0b45f252c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:59.271411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:59.287309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] MFA Rules not processed for user `4690104f5bea480d87b7c43488182441`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:06:59.287309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:59.382825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1106/2193] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:06:59.397379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1088/2194] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:06:59.401388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:59.402067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:59.403401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:59.403934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:06:59.534231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed313ab0-5f5c-436c-a031-bf8f4ccecc54 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:59.546782 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7368d2-3aa5-435a-b9ca-71d767e968f0 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:59.553501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:59.554624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:59.555081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:59.555554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:59.556369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:59.556476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04917af-477a-49de-a796-67b307017d57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04084ffe1b0548fea75ea144977e4f15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:59.557185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04917af-477a-49de-a796-67b307017d57 service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:59.557477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04917af-477a-49de-a796-67b307017d57 service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:59.557775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04917af-477a-49de-a796-67b307017d57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:59.559216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:59.576576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04917af-477a-49de-a796-67b307017d57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:59.579185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04917af-477a-49de-a796-67b307017d57 service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:59.593336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1107/2195] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:06:59.684023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1089/2196] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:06:59 2022] POST /identity/v3/projects => generated 371 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:59.687454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04917af-477a-49de-a796-67b307017d57 service glance] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:59.708974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04917af-477a-49de-a796-67b307017d57 service glance] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:59.791194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:59.792415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:59.792960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:59.793961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:59.794617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:59.797167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:59.930734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1108/2197] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:06:59 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:06:59.934549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:59.944522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61ea9-5860-44c7-af7e-b9c9187d8a6a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:59.949663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:59.950300 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:59.950577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:59.950849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:06:59.951486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:06:59.953477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:06:59.964765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1090/2198] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:06:59.969093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:06:59.979606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb87678-8f73-4ccf-8c2e-3caae451b6d2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:06:59.985687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:06:59.986729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:06:59.987013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:06:59.987295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] PATH_INFO: `/v3/projects/686c7aa2d3384c8a890065be02c5078f/users/3372ac9db1e649a898e15c3bf664fc3b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:00.004725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=686c7aa2d3384c8a890065be02c5078f, user_id=3372ac9db1e649a898e15c3bf664fc3b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:00.007413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:00.097728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:00.104006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1109/2199] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:06:59 2022] PUT /identity/v3/projects/686c7aa2d3384c8a890065be02c5078f/users/3372ac9db1e649a898e15c3bf664fc3b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:00.108205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:00.138149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61119bbd-f466-45dc-ae73-ac33e0d935ac admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:00.217622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:00.218559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:00.218864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:00.219137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:00.219909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:00.222173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:00.252134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1091/2200] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:00.256767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:00.269877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d95d-99f7-420d-93bf-c30f4d9a0805 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:00.280455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83096c-b52c-4126-92e4-909b663d0e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:00.280455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83096c-b52c-4126-92e4-909b663d0e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:00.280455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83096c-b52c-4126-92e4-909b663d0e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:00.280455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83096c-b52c-4126-92e4-909b663d0e6d admin admin] PATH_INFO: `/v3/projects/686c7aa2d3384c8a890065be02c5078f/users/3372ac9db1e649a898e15c3bf664fc3b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:00.281627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83096c-b52c-4126-92e4-909b663d0e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=686c7aa2d3384c8a890065be02c5078f, user_id=3372ac9db1e649a898e15c3bf664fc3b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:00.285257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83096c-b52c-4126-92e4-909b663d0e6d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:00.297590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1110/2201] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:07:00 2022] PUT /identity/v3/projects/686c7aa2d3384c8a890065be02c5078f/users/3372ac9db1e649a898e15c3bf664fc3b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:00.404071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:01.253472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:01.255683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:01.290839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cea4e-3a7f-480a-a06b-a20163881c4b admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:01.485154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:01.485864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:01.486146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:01.486414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] PATH_INFO: `/v3/users/2c58d4800ba54bc58779466339406fab` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:01.494433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c58d4800ba54bc58779466339406fab)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:01.500121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:01.839213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c58d4800ba54bc58779466339406fab'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:01.840208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c58d4800ba54bc58779466339406fab'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:01.852095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c58d4800ba54bc58779466339406fab'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:01.874885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c58d4800ba54bc58779466339406fab'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:01.878407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c58d4800ba54bc58779466339406fab'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:01.993987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1092/2202] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:01 2022] DELETE /identity/v3/users/2c58d4800ba54bc58779466339406fab => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:01.995576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:01.996151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:01.996505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:02.061041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:02.062339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:02.123149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:02.151616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:02.153147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:02.156682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1111/2203] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:07:02.169766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:02.229689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bab3c-1d49-452a-859c-82a9c7e76597 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:02.241717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:02.242582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:02.243604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:02.243909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:02.271030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:02.272658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:02.285586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1093/2204] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:03.263203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:03.264842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:03.299680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd903b0f-de5b-4aa0-a156-6404ec86a029 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:03.314510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:03.315184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:03.315459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:03.315738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] PATH_INFO: `/v3/projects/189f2c9a669a46f68369bb3c6b058f8f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:03.322706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=189f2c9a669a46f68369bb3c6b058f8f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:03.322706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:03.561557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189f2c9a669a46f68369bb3c6b058f8f', 'request_id': 'req-967b1abf-8ca6-44ff-8366-947d524ebb9b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:03.638856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189f2c9a669a46f68369bb3c6b058f8f', 'request_id': 'req-967b1abf-8ca6-44ff-8366-947d524ebb9b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:03.646381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189f2c9a669a46f68369bb3c6b058f8f', 'request_id': 'req-967b1abf-8ca6-44ff-8366-947d524ebb9b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:03.664888 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] The token cache is being invalidate because project 189f2c9a669a46f68369bb3c6b058f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:03.665527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:03.669499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:03.675369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1112/2205] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:03 2022] DELETE /identity/v3/projects/189f2c9a669a46f68369bb3c6b058f8f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:03.681251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:03.719817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370b803-9195-4eb4-b962-d4952e69fefc service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:03.844402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:03.845261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:03.846041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:03.847295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] PATH_INFO: `/v3/users/24a2862a21644384bfec1539eeb651b6` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:03.848902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a2862a21644384bfec1539eeb651b6)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:03.851205 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:04.096102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a2862a21644384bfec1539eeb651b6'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:04.096102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a2862a21644384bfec1539eeb651b6'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:04.109416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a2862a21644384bfec1539eeb651b6'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:04.149415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a2862a21644384bfec1539eeb651b6'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:04.150364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a2862a21644384bfec1539eeb651b6'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:04.285344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1094/2206] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:03 2022] DELETE /identity/v3/users/24a2862a21644384bfec1539eeb651b6 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:05.401111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:05.406001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:05.435580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b1abf-8ca6-44ff-8366-947d524ebb9b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:05.509075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:05.509719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:05.510024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:05.510337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] PATH_INFO: `/v3/projects/b7d1cbe564354e6e89a2747fe4121e4b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:05.513125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d1cbe564354e6e89a2747fe4121e4b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:05.515257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:05.707962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d1cbe564354e6e89a2747fe4121e4b', 'request_id': 'req-a16531ab-d9b6-421a-b17b-b575469e2add'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:05.790764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d1cbe564354e6e89a2747fe4121e4b', 'request_id': 'req-a16531ab-d9b6-421a-b17b-b575469e2add'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:05.801010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d1cbe564354e6e89a2747fe4121e4b', 'request_id': 'req-a16531ab-d9b6-421a-b17b-b575469e2add'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:05.812996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] The token cache is being invalidate because project b7d1cbe564354e6e89a2747fe4121e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:05.813430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:05.814103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:05.815713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1113/2207] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:05 2022] DELETE /identity/v3/projects/b7d1cbe564354e6e89a2747fe4121e4b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:05.820051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:05.821962 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:05.858390 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dd9ab-6d4a-41a2-bc0c-67690165b926 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:05.929458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:06.010541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:06.011598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:06.012094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:06.012674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] PATH_INFO: `/v3/users/1b09ed7e0a5549678994a9e5aa72945a` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:06.014173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b09ed7e0a5549678994a9e5aa72945a)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:06.017631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:06.294088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b09ed7e0a5549678994a9e5aa72945a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:06.295662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b09ed7e0a5549678994a9e5aa72945a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:06.312255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b09ed7e0a5549678994a9e5aa72945a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:06.353144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b09ed7e0a5549678994a9e5aa72945a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:06.358246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c0c54252-1469-42d8-8fa7-2877341ef6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b09ed7e0a5549678994a9e5aa72945a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:06.450958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1095/2208] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:05 2022] DELETE /identity/v3/users/1b09ed7e0a5549678994a9e5aa72945a => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:07.301298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:07.302848 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:07.341300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16531ab-d9b6-421a-b17b-b575469e2add admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:07.346257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:07.412657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:07.413367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:07.413649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:07.416759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] PATH_INFO: `/v3/projects/a94599aa400a463c95706fefaeb4f70f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:07.416759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94599aa400a463c95706fefaeb4f70f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:07.419016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:07.566056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94599aa400a463c95706fefaeb4f70f', 'request_id': 'req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:07.668356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94599aa400a463c95706fefaeb4f70f', 'request_id': 'req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:07.678650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94599aa400a463c95706fefaeb4f70f', 'request_id': 'req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:07.708970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] The token cache is being invalidate because project a94599aa400a463c95706fefaeb4f70f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:07.709714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:07.710813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-23f02bb7-3f26-4af7-8e7c-8393fcab07ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:07.715770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1114/2209] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:07 2022] DELETE /identity/v3/projects/a94599aa400a463c95706fefaeb4f70f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:07.720996 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c254cf-f77c-45fe-b310-7899b97d3f17 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:07.721520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c254cf-f77c-45fe-b310-7899b97d3f17 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:07.721520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c254cf-f77c-45fe-b310-7899b97d3f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:07.752411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d4c254cf-f77c-45fe-b310-7899b97d3f17 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:07.753898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d4c254cf-f77c-45fe-b310-7899b97d3f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:07.846494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1096/2210] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:07.854543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb48e-0271-4155-b815-dcf681fdbb8e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:07.854835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb48e-0271-4155-b815-dcf681fdbb8e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:07.855173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb48e-0271-4155-b815-dcf681fdbb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:07.912107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cc6cb48e-0271-4155-b815-dcf681fdbb8e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:07.913820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cc6cb48e-0271-4155-b815-dcf681fdbb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:07.946622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1115/2211] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:07.951440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c254cf-f77c-45fe-b310-7899b97d3f17 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:07.964735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c254cf-f77c-45fe-b310-7899b97d3f17 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:07.973990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:07.974907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:07.978596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:07.979109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:07.980124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:07:07.980643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:07.982578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:08.000642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1097/2212] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:07:08.001792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb48e-0271-4155-b815-dcf681fdbb8e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.013041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb48e-0271-4155-b815-dcf681fdbb8e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:08.019003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:08.019643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.019931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.020211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.020893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:08.022842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:08.111750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1116/2213] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:07:08 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:07:08.113301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.126602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc292379-858d-4c38-84d8-64fcceab2e44 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:08.194306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:08.248458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:08.250248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.250752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.251208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.252526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:08.254810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:08.359124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.360837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1098/2214] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:07:08 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:07:08.380545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a923-e20e-481a-a9e9-a08f7d25f265 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:08.387675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:08.388308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.388587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.388860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.389607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:08.391350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:08.402897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1117/2215] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:08.406974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.424923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e03937-4073-4d7f-ae1d-6769ea665fcf admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:08.432681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c7ee2-99a2-4872-a664-b0d7d9435eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:08.433369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7ee2-99a2-4872-a664-b0d7d9435eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.433641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7ee2-99a2-4872-a664-b0d7d9435eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.433951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7ee2-99a2-4872-a664-b0d7d9435eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.434604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c7ee2-99a2-4872-a664-b0d7d9435eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:08.437111 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c7ee2-99a2-4872-a664-b0d7d9435eb9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:08.450398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1099/2216] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:07:08.454626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.471258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d4323-ec79-490a-9ada-00a1ad6af755 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:08.503482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa8801-2c26-4596-bf0a-6dfa3fa7d6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:08.504179 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8801-2c26-4596-bf0a-6dfa3fa7d6f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.504464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8801-2c26-4596-bf0a-6dfa3fa7d6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.504740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8801-2c26-4596-bf0a-6dfa3fa7d6f8 admin admin] PATH_INFO: `/v3/projects/2be7916bd41644d4af8d26e92d80b4ee/users/9d2d3845283743fe8eadddfcd56e36f0/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.550066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa8801-2c26-4596-bf0a-6dfa3fa7d6f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be7916bd41644d4af8d26e92d80b4ee, user_id=9d2d3845283743fe8eadddfcd56e36f0, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:08.554606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa8801-2c26-4596-bf0a-6dfa3fa7d6f8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:08.590436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.590754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.590873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.624431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] MFA Rules not processed for user `3372ac9db1e649a898e15c3bf664fc3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:08.625839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:08.646664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1118/2217] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:07:08 2022] PUT /identity/v3/projects/2be7916bd41644d4af8d26e92d80b4ee/users/9d2d3845283743fe8eadddfcd56e36f0/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:08.657243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feacd46-ea4c-4b68-87a6-f7e4f4b73901 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.657537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feacd46-ea4c-4b68-87a6-f7e4f4b73901 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.658187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feacd46-ea4c-4b68-87a6-f7e4f4b73901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.693165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8feacd46-ea4c-4b68-87a6-f7e4f4b73901 None None] MFA Rules not processed for user `9d2d3845283743fe8eadddfcd56e36f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:08.695163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8feacd46-ea4c-4b68-87a6-f7e4f4b73901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:08.715864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1100/2218] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:07:08.736067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.741186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:08.776639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1119/2219] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:07:08.781502 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feacd46-ea4c-4b68-87a6-f7e4f4b73901 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.827736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d4b3c-d2d7-473a-b51a-677ccc227858 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:08.864283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feacd46-ea4c-4b68-87a6-f7e4f4b73901 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:08.869272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:08.869911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.870197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.870469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.871136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:08.873097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:08.902819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:08.903558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:08.903837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:08.904107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:08.906019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:08.967948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1120/2220] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:07:08 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:07:08.968995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:08.978956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:08.981108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:09.001893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1101/2221] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:09.048325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911f8ae-57b8-4ab1-b2ad-f5d2ab9812b0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:09.053500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:09.054148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:09.054425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:09.054689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:09.055872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:09.057790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:09.262542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1121/2222] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:07:08 2022] POST /identity/v3/users => generated 292 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:07:09.270094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:09.281118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ce412-5588-41c4-9d99-d463069f79ff service nova] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:09.290081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:09.290853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:09.291138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:09.291470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:09.292064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:09.294053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:09.322553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:09.332478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1102/2223] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:09.339123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:09.345665 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c1683-52e5-44ff-80dd-af4d7f3fda83 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:09.355508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844defa-f995-4cc1-9947-dd521d52bac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:09.356203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844defa-f995-4cc1-9947-dd521d52bac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:09.356420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844defa-f995-4cc1-9947-dd521d52bac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:09.356697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844defa-f995-4cc1-9947-dd521d52bac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:09.357354 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844defa-f995-4cc1-9947-dd521d52bac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:09.359147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844defa-f995-4cc1-9947-dd521d52bac8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:09.387907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1122/2224] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:09.397759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:09.426398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70773b0-5268-4f08-9c01-275bf47be2ad admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:09.436153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:09.444760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:09.444760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:09.444760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] PATH_INFO: `/v3/projects/c310c23f0ae3436b9e96d1fcd573f813/users/530d7c9714b84bb4ae5415cd29a57d48/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:09.472746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c310c23f0ae3436b9e96d1fcd573f813, user_id=530d7c9714b84bb4ae5415cd29a57d48, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:09.479886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:09.572096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1103/2225] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:07:09 2022] PUT /identity/v3/projects/c310c23f0ae3436b9e96d1fcd573f813/users/530d7c9714b84bb4ae5415cd29a57d48/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:09.684608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f110b-d61d-4669-9c4f-2e2a75a7689c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:09.684608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f110b-d61d-4669-9c4f-2e2a75a7689c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:09.684842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f110b-d61d-4669-9c4f-2e2a75a7689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:09.737487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-093f110b-d61d-4669-9c4f-2e2a75a7689c None None] MFA Rules not processed for user `530d7c9714b84bb4ae5415cd29a57d48`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:09.739127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-093f110b-d61d-4669-9c4f-2e2a75a7689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:09.841642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1123/2226] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:09.842523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:09.920337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88caaa-eae9-4922-a966-9d0f3fc45296 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:09.925350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:09.926066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:09.926414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:09.927124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:09.932349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:09.938356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:10.053342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1104/2227] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:07:09 2022] POST /identity/v3/projects => generated 383 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:07:10.057794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f110b-d61d-4669-9c4f-2e2a75a7689c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:10.068600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f110b-d61d-4669-9c4f-2e2a75a7689c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:10.134864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:10.135704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:10.136038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:10.136910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:10.137704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:10.140021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:10.199752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:10.262267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1124/2228] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:07:10 2022] POST /identity/v3/users => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:07:10.263442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:10.277012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b103f2-6a79-418f-84fe-701fd3a2f088 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:10.284925 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:10.285886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:10.286282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:10.287171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:10.287818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:10.290083 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:10.308831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1105/2229] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:10.313349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:10.335334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea1644-1030-4076-89f6-03eb7000ad6a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:10.344244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:10.344931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:10.345208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:10.345480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] PATH_INFO: `/v3/projects/315a2ff743584ed7bffc6e6bcf001194/users/938fdeaf49c644dc8c1b7c9fb0315c1b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:10.369681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315a2ff743584ed7bffc6e6bcf001194, user_id=938fdeaf49c644dc8c1b7c9fb0315c1b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:10.374650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:10.477174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1125/2230] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:07:10 2022] PUT /identity/v3/projects/315a2ff743584ed7bffc6e6bcf001194/users/938fdeaf49c644dc8c1b7c9fb0315c1b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:10.478352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:10.499531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ba1c7-a9e0-4cc1-ba99-0a9a54c97988 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:10.565234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb6251-e0ef-4ef9-ad94-629dff9bb80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:10.565795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb6251-e0ef-4ef9-ad94-629dff9bb80b admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:10.566172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb6251-e0ef-4ef9-ad94-629dff9bb80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:10.566627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb6251-e0ef-4ef9-ad94-629dff9bb80b admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:10.567232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb6251-e0ef-4ef9-ad94-629dff9bb80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:10.579586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb6251-e0ef-4ef9-ad94-629dff9bb80b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:10.601170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1106/2231] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:10.605262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:10.619706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d16472-7745-4566-8229-4d001b7aa886 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:10.642341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:10.643309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:10.643592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:10.643873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] PATH_INFO: `/v3/projects/315a2ff743584ed7bffc6e6bcf001194/users/938fdeaf49c644dc8c1b7c9fb0315c1b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:10.649167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=315a2ff743584ed7bffc6e6bcf001194, user_id=938fdeaf49c644dc8c1b7c9fb0315c1b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:10.651954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:10.677378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1126/2232] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:07:10 2022] PUT /identity/v3/projects/315a2ff743584ed7bffc6e6bcf001194/users/938fdeaf49c644dc8c1b7c9fb0315c1b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:10.685945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aa8e6-d990-438e-89a9-b787b998e8f0 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:10.686329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aa8e6-d990-438e-89a9-b787b998e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:10.686701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aa8e6-d990-438e-89a9-b787b998e8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:10.720726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-171aa8e6-d990-438e-89a9-b787b998e8f0 None None] MFA Rules not processed for user `938fdeaf49c644dc8c1b7c9fb0315c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:10.722084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-171aa8e6-d990-438e-89a9-b787b998e8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:10.805343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1107/2233] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:10.819077 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:10.821089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:10.931398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b0113-8ae6-4d74-a8c9-5d957bb219aa admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:10.938321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:10.938966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:10.939257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:10.939558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:11.017451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:11.017451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:11.026458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1127/2234] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:11.177568 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:11.462057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:12.750413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e27799-d597-418d-81bf-19d3a32325f7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:12.751352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e27799-d597-418d-81bf-19d3a32325f7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:12.751631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e27799-d597-418d-81bf-19d3a32325f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:12.794656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-25e27799-d597-418d-81bf-19d3a32325f7 None None] MFA Rules not processed for user `4690104f5bea480d87b7c43488182441`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:12.796004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-25e27799-d597-418d-81bf-19d3a32325f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:12.869049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1108/2235] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:12.882025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:13.423250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:13.425441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:13.439149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40936fd9-4dcc-4893-a299-336f04fe1946 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:13.442994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47737975-9b86-45b0-bfd6-f27c01cdb3f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:13.443885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737975-9b86-45b0-bfd6-f27c01cdb3f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:13.444295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737975-9b86-45b0-bfd6-f27c01cdb3f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:13.444570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737975-9b86-45b0-bfd6-f27c01cdb3f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:13.539692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47737975-9b86-45b0-bfd6-f27c01cdb3f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:13.541346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47737975-9b86-45b0-bfd6-f27c01cdb3f1 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:13.565279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1128/2236] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:15.475656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e27799-d597-418d-81bf-19d3a32325f7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:15.489460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e27799-d597-418d-81bf-19d3a32325f7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:15.495867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:15.496392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:15.496673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:15.497000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:15.514439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:15.516142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:15.531222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1109/2237] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:18.485770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b8179-2884-45af-97a3-50fd5abeb0bc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:18.486236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b8179-2884-45af-97a3-50fd5abeb0bc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:18.486722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b8179-2884-45af-97a3-50fd5abeb0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:18.513894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7d4b8179-2884-45af-97a3-50fd5abeb0bc None None] MFA Rules not processed for user `8c688fff323943b3bbcbb0569d974d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:18.515424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7d4b8179-2884-45af-97a3-50fd5abeb0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:18.591897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1129/2238] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:20.639877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:20.676781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:20.720097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a7a07-1301-46c3-a562-c7434805d7eb service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:20.727834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:20.728679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:20.729040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:20.729317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] PATH_INFO: `/v3/users/9d2d3845283743fe8eadddfcd56e36f0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:20.730731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2d3845283743fe8eadddfcd56e36f0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:20.732762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:20.952859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2d3845283743fe8eadddfcd56e36f0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:20.953596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2d3845283743fe8eadddfcd56e36f0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:20.962195 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2d3845283743fe8eadddfcd56e36f0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:20.977025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2d3845283743fe8eadddfcd56e36f0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:20.978121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2d3845283743fe8eadddfcd56e36f0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:21.108651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1110/2239] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:20 2022] DELETE /identity/v3/users/9d2d3845283743fe8eadddfcd56e36f0 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:21.113069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:21.113552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:21.113675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:21.136432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:21.138183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:21.203979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1130/2240] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:07:21.214061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:21.215478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:21.275394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc701a9b-a750-43fe-8d87-d17e316ca348 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:21.279827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:21.280596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:21.280982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:21.281333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:21.290874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:21.292302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:21.306109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1111/2241] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:21.783490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:21.784904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:21.798342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8d88-eec8-4bde-b85c-3ca3f9aee6d3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:21.803502 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:21.804136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:21.804456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:21.808253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] PATH_INFO: `/v3/projects/2be7916bd41644d4af8d26e92d80b4ee` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:21.808253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be7916bd41644d4af8d26e92d80b4ee)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:21.808253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:21.809419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:21.971367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be7916bd41644d4af8d26e92d80b4ee', 'request_id': 'req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.050332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be7916bd41644d4af8d26e92d80b4ee', 'request_id': 'req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.057582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be7916bd41644d4af8d26e92d80b4ee', 'request_id': 'req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.069402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] The token cache is being invalidate because project 2be7916bd41644d4af8d26e92d80b4ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:22.069626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.070388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.075171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1131/2242] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:21 2022] DELETE /identity/v3/projects/2be7916bd41644d4af8d26e92d80b4ee => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:22.077628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:22.090585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950c0df-b3d6-448c-a9ab-3eb15a946230 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:22.142402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:22.153899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:22.154607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:22.154885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:22.155154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] PATH_INFO: `/v3/users/530d7c9714b84bb4ae5415cd29a57d48` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:22.156311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] RBAC: Authorizing `identity:delete_user(user_id=530d7c9714b84bb4ae5415cd29a57d48)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:22.158347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:22.341706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530d7c9714b84bb4ae5415cd29a57d48'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.342298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530d7c9714b84bb4ae5415cd29a57d48'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.349181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530d7c9714b84bb4ae5415cd29a57d48'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.362625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530d7c9714b84bb4ae5415cd29a57d48'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.363296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530d7c9714b84bb4ae5415cd29a57d48'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:22.453477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1112/2243] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:22 2022] DELETE /identity/v3/users/530d7c9714b84bb4ae5415cd29a57d48 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:22.845958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:22.847285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:22.871489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe786c2-4c9b-4713-8b60-bb4bd6741ed8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:22.942319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:22.942942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:22.943222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:22.943511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] PATH_INFO: `/v3/projects/c310c23f0ae3436b9e96d1fcd573f813` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:22.944628 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c310c23f0ae3436b9e96d1fcd573f813)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:22.947140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:23.083886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c310c23f0ae3436b9e96d1fcd573f813', 'request_id': 'req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.219377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c310c23f0ae3436b9e96d1fcd573f813', 'request_id': 'req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.227101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c310c23f0ae3436b9e96d1fcd573f813', 'request_id': 'req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.238233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] The token cache is being invalidate because project c310c23f0ae3436b9e96d1fcd573f813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:23.238635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.239307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.241271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1132/2244] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:22 2022] DELETE /identity/v3/projects/c310c23f0ae3436b9e96d1fcd573f813 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:23.245833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:23.249600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:23.263707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bf9bf-a89e-44cb-9133-0dc1ecd4850d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:23.338385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:23.339023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:23.339304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:23.339574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] PATH_INFO: `/v3/users/938fdeaf49c644dc8c1b7c9fb0315c1b` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:23.340729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=938fdeaf49c644dc8c1b7c9fb0315c1b)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:23.342709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:23.533122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938fdeaf49c644dc8c1b7c9fb0315c1b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.533688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938fdeaf49c644dc8c1b7c9fb0315c1b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.541025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938fdeaf49c644dc8c1b7c9fb0315c1b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.552733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:23.554244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938fdeaf49c644dc8c1b7c9fb0315c1b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.555974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3d86baa3-d81a-467c-a1ae-05935d3d5916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938fdeaf49c644dc8c1b7c9fb0315c1b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:23.651654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1113/2245] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:23 2022] DELETE /identity/v3/users/938fdeaf49c644dc8c1b7c9fb0315c1b => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:07:24.073607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:24.075100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:24.091106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f5f9e-c1d3-4ab6-a6c6-bef017e9254f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:24.118031 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:24.161529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:24.162198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.162477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.162747 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] PATH_INFO: `/v3/projects/315a2ff743584ed7bffc6e6bcf001194` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:24.164540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] RBAC: Authorizing `identity:delete_project(project_id=315a2ff743584ed7bffc6e6bcf001194)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:24.166569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:24.281157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315a2ff743584ed7bffc6e6bcf001194', 'request_id': 'req-3601c261-0e51-4b27-92bc-0563664d667c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:24.367885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315a2ff743584ed7bffc6e6bcf001194', 'request_id': 'req-3601c261-0e51-4b27-92bc-0563664d667c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:24.375615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315a2ff743584ed7bffc6e6bcf001194', 'request_id': 'req-3601c261-0e51-4b27-92bc-0563664d667c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:24.388341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] The token cache is being invalidate because project 315a2ff743584ed7bffc6e6bcf001194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:24.388654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:24.389708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-3601c261-0e51-4b27-92bc-0563664d667c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:24.391907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1133/2246] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:24 2022] DELETE /identity/v3/projects/315a2ff743584ed7bffc6e6bcf001194 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:24.397073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002d20-afba-4f2e-a5b9-482130b67101 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.397538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002d20-afba-4f2e-a5b9-482130b67101 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.398539 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002d20-afba-4f2e-a5b9-482130b67101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:24.421187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-82002d20-afba-4f2e-a5b9-482130b67101 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:24.422484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-82002d20-afba-4f2e-a5b9-482130b67101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:24.514430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1114/2247] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:24.523864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd25c0e-70bb-40e8-961d-6cd3adb6ba25 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.524148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd25c0e-70bb-40e8-961d-6cd3adb6ba25 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.524440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd25c0e-70bb-40e8-961d-6cd3adb6ba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:24.558434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-afd25c0e-70bb-40e8-961d-6cd3adb6ba25 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:24.559292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-afd25c0e-70bb-40e8-961d-6cd3adb6ba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:24.567739 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1134/2248] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:24.572799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002d20-afba-4f2e-a5b9-482130b67101 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:24.583825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002d20-afba-4f2e-a5b9-482130b67101 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:24.589524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:24.590061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.590344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.590618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:24.591653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:07:24.591926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:24.593769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:24.611827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1115/2249] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:07:24.613171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd25c0e-70bb-40e8-961d-6cd3adb6ba25 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:24.625691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd25c0e-70bb-40e8-961d-6cd3adb6ba25 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:24.632360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:24.632360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.632360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.632360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:24.636113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:24.639828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:24.750583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1135/2250] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:07:24 2022] POST /identity/v3/projects => generated 389 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:07:24.754786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:24.764883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0223b-9ab9-4da4-9e8d-13e33570be29 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:24.824354 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:24.824959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.825239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.825570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:24.826246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:24.828190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:24.936764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1116/2251] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:07:24 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:07:24.944425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:24.956815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d62c7c-4c0f-4c33-96fe-b884d258fb74 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:24.961897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:24.962524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.962798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.963067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:24.963707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:24.965275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:24.975689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1136/2252] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:07:24.980267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:24.992081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb833bf2-f0ad-4046-aa31-2e7219e75699 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:24.998333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff7ea3-3f35-4013-93e0-1160f4c5add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:24.998979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff7ea3-3f35-4013-93e0-1160f4c5add0 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:24.999389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff7ea3-3f35-4013-93e0-1160f4c5add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:24.999672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff7ea3-3f35-4013-93e0-1160f4c5add0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:25.000497 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff7ea3-3f35-4013-93e0-1160f4c5add0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:25.002129 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff7ea3-3f35-4013-93e0-1160f4c5add0 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:25.017133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1117/2253] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:25.025101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:25.036561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2ff12-b5f4-4bcd-a8ce-7766b2e32fc3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:25.042666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:25.043710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:25.043994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:25.044274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] PATH_INFO: `/v3/projects/af92d765bb814bc1929b8e5f08a02acd/users/51daab050ff8463ca882141d0fe01b8d/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:25.059859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af92d765bb814bc1929b8e5f08a02acd, user_id=51daab050ff8463ca882141d0fe01b8d, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:25.062662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:25.149833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1137/2254] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:07:25 2022] PUT /identity/v3/projects/af92d765bb814bc1929b8e5f08a02acd/users/51daab050ff8463ca882141d0fe01b8d/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:25.160166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e4a38-d707-463f-8bb1-f954dd1404b7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:25.160447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e4a38-d707-463f-8bb1-f954dd1404b7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:25.160724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e4a38-d707-463f-8bb1-f954dd1404b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:25.196957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-b00e4a38-d707-463f-8bb1-f954dd1404b7 None None] MFA Rules not processed for user `51daab050ff8463ca882141d0fe01b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:25.198227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b00e4a38-d707-463f-8bb1-f954dd1404b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:25.252719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:25.274952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1118/2255] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:25.292522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:25.296940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:25.369771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:25.375718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f39f30-d29d-46b7-a3a7-0b237bd11993 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:25.380618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:25.383052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:25.383052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:25.383052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:25.392337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:25.393972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:25.416350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1138/2256] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:25.588047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5209c-236e-4717-b8b3-47d3b7508301 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:25.588606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5209c-236e-4717-b8b3-47d3b7508301 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:25.588769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5209c-236e-4717-b8b3-47d3b7508301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:25.608631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-c5b5209c-236e-4717-b8b3-47d3b7508301 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:25.609973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c5b5209c-236e-4717-b8b3-47d3b7508301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:25.693393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1119/2257] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:07:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:25.746692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:25.750681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:25.853146 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a848aba-38de-4512-944b-72c4ccbc8210 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:25.858090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5eab1-efcc-41f0-a7cc-16d750d18ed2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:25.858697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5eab1-efcc-41f0-a7cc-16d750d18ed2 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:25.858975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5eab1-efcc-41f0-a7cc-16d750d18ed2 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:25.859246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5eab1-efcc-41f0-a7cc-16d750d18ed2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:25.867630 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5eab1-efcc-41f0-a7cc-16d750d18ed2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:25.869198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5eab1-efcc-41f0-a7cc-16d750d18ed2 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:25.881064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1139/2258] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:07:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:07:35.686272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=32 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:35.883020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=37 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:50.259080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5209c-236e-4717-b8b3-47d3b7508301 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:50.341756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5209c-236e-4717-b8b3-47d3b7508301 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:50.347265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:50.348282 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:50.348651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:50.349001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] PATH_INFO: `/v3/users/51daab050ff8463ca882141d0fe01b8d` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:50.352029 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51daab050ff8463ca882141d0fe01b8d)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:50.354834 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:50.651919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51daab050ff8463ca882141d0fe01b8d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:50.653133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51daab050ff8463ca882141d0fe01b8d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:50.671548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51daab050ff8463ca882141d0fe01b8d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:50.694883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51daab050ff8463ca882141d0fe01b8d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:50.697049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51daab050ff8463ca882141d0fe01b8d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:50.822957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1120/2259] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:50 2022] DELETE /identity/v3/users/51daab050ff8463ca882141d0fe01b8d => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:50.828041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcf53-af3e-452a-bafd-889da537a973 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:50.828530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcf53-af3e-452a-bafd-889da537a973 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:50.828672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcf53-af3e-452a-bafd-889da537a973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:50.853331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-d70fcf53-af3e-452a-bafd-889da537a973 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:50.855022 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d70fcf53-af3e-452a-bafd-889da537a973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:50.953902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1140/2260] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:50.964372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:50.966890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.038445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2229-9502-4e89-aae6-81a66e9af69f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.040102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2229-9502-4e89-aae6-81a66e9af69f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.040102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2229-9502-4e89-aae6-81a66e9af69f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.040904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1141/2261] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:07:51.046864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734696-7597-4408-9141-2354ee3b1088 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.047145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734696-7597-4408-9141-2354ee3b1088 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.047561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734696-7597-4408-9141-2354ee3b1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.068466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d1d48-0385-4024-8a37-fa98ac19d1b7 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:51.073396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:51.073541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-60734696-7597-4408-9141-2354ee3b1088 None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:51.074386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.074857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.075071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-60734696-7597-4408-9141-2354ee3b1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.075682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.087213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:51.088939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:51.114939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1121/2262] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:51.196308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1142/2263] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 13 13:07:51.204929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:51.206718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.216821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60734696-7597-4408-9141-2354ee3b1088 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:51.218454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-60734696-7597-4408-9141-2354ee3b1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.233120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60734696-7597-4408-9141-2354ee3b1088 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:51.238245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:51.239346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.239636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.239918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] PATH_INFO: `/v3/users/ad0978e6967348668bf9f0b45f252c5c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.242122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0978e6967348668bf9f0b45f252c5c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:51.243989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:51.244438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:51.292159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59cd4-a7bc-4209-909a-05b0d7d07ca1 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:51.297991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6300b-7530-46e5-9402-162aa5d36c38 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04084ffe1b0548fea75ea144977e4f15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:51.299984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6300b-7530-46e5-9402-162aa5d36c38 service glance] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.299984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6300b-7530-46e5-9402-162aa5d36c38 service glance] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.299984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6300b-7530-46e5-9402-162aa5d36c38 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.313989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6300b-7530-46e5-9402-162aa5d36c38 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:51.315536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6300b-7530-46e5-9402-162aa5d36c38 service glance] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:51.327937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1122/2264] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:51.506984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0978e6967348668bf9f0b45f252c5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:51.507861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0978e6967348668bf9f0b45f252c5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:51.523672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0978e6967348668bf9f0b45f252c5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:51.547048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0978e6967348668bf9f0b45f252c5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:51.547777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a1d26ce7-1c87-4e12-b524-552a96f48449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0978e6967348668bf9f0b45f252c5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:51.594010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03043-96fd-4694-987d-7ce21209168b None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.594333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03043-96fd-4694-987d-7ce21209168b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.594569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03043-96fd-4694-987d-7ce21209168b None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.596404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1123/2265] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:07:51.602153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.602440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.603709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.627744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] MFA Rules not processed for user `4ed50a15319e4991ae90b0e5271f4c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:51.629216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.641582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1124/2266] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:07:51.650002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:51.650002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.655889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1143/2267] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:07:51 2022] DELETE /identity/v3/users/ad0978e6967348668bf9f0b45f252c5c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:51.659977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.663307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.663307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.669326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0eb4d0-49c2-4726-b543-caa52bf3121c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:51.674147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:51.687674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:51.689067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.778322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1144/2268] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:07:51.779821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:51.781768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.799964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:51.800660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.800952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.801275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] PATH_INFO: `/v3/projects/af92d765bb814bc1929b8e5f08a02acd` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.802531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] RBAC: Authorizing `identity:delete_project(project_id=af92d765bb814bc1929b8e5f08a02acd)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:51.806631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:51.874041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49636bd-dea8-4a69-a108-abb824513666 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:51.877889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:51.878660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.878957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.879213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.887619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:51.889087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:51.898411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1145/2269] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:51.899359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:51.900955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:51.914755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199c9db-609e-48af-90e8-0e3692b442a0 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:51.918917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c83a7-00c7-44cc-b7f0-cf3177a1aa3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:51.935817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c83a7-00c7-44cc-b7f0-cf3177a1aa3f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.935817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c83a7-00c7-44cc-b7f0-cf3177a1aa3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:51.935817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c83a7-00c7-44cc-b7f0-cf3177a1aa3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:51.935817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c83a7-00c7-44cc-b7f0-cf3177a1aa3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:51.935817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c83a7-00c7-44cc-b7f0-cf3177a1aa3f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:51.944581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1146/2270] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:51.985193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af92d765bb814bc1929b8e5f08a02acd', 'request_id': 'req-7fed9dfe-643e-4763-94b1-97d9920e137f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:51.995801 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13466-3c9c-40c6-8c14-87989295055c None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:51.996166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13466-3c9c-40c6-8c14-87989295055c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:52.000393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13466-3c9c-40c6-8c14-87989295055c None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:52.000393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1147/2271] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:07:52.004483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:52.004849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:52.005195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:52.029045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:52.029317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:52.048078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af92d765bb814bc1929b8e5f08a02acd', 'request_id': 'req-7fed9dfe-643e-4763-94b1-97d9920e137f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:52.058437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af92d765bb814bc1929b8e5f08a02acd', 'request_id': 'req-7fed9dfe-643e-4763-94b1-97d9920e137f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:52.072076 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] The token cache is being invalidate because project af92d765bb814bc1929b8e5f08a02acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:52.072681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:52.073318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:52.094456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1125/2272] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:51 2022] DELETE /identity/v3/projects/af92d765bb814bc1929b8e5f08a02acd => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:07:52.145089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1148/2273] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:07:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:07:52.159298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:52.160965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:52.298920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:52.304065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed9dfe-643e-4763-94b1-97d9920e137f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:52.309389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cedce-f229-4a3d-abca-c92be55b9040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:52.310204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cedce-f229-4a3d-abca-c92be55b9040 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:52.310480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cedce-f229-4a3d-abca-c92be55b9040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:52.310749 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cedce-f229-4a3d-abca-c92be55b9040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:52.322867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0cedce-f229-4a3d-abca-c92be55b9040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:52.324397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0cedce-f229-4a3d-abca-c92be55b9040 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:52.341328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1126/2274] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:07:52.917945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:52.919554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:52.962230 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:52.996511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6c06a-1865-40d7-a8fd-2c8717ffe733 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:53.001028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:53.001661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.001974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.002422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] PATH_INFO: `/v3/projects/743eb3a5068f4b59bc85e7fb17c83787` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.003432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=743eb3a5068f4b59bc85e7fb17c83787)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:53.005335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:53.128187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743eb3a5068f4b59bc85e7fb17c83787', 'request_id': 'req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:53.228239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743eb3a5068f4b59bc85e7fb17c83787', 'request_id': 'req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:53.238772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743eb3a5068f4b59bc85e7fb17c83787', 'request_id': 'req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:53.249611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] The token cache is being invalidate because project 743eb3a5068f4b59bc85e7fb17c83787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:07:53.250157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:53.250956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a6a79f7-1d3a-42bd-ab53-e482563ec1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:07:53.252819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1149/2275] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:07:52 2022] DELETE /identity/v3/projects/743eb3a5068f4b59bc85e7fb17c83787 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:53.258503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8759-9f4a-45a7-afbb-4ed8aefa42b4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.258789 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8759-9f4a-45a7-afbb-4ed8aefa42b4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.259061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8759-9f4a-45a7-afbb-4ed8aefa42b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.291312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d8bf8759-9f4a-45a7-afbb-4ed8aefa42b4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:53.292810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d8bf8759-9f4a-45a7-afbb-4ed8aefa42b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:53.370182 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1127/2276] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:53.378907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccb54c-b698-460a-9637-bab45ac0052a None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.379717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccb54c-b698-460a-9637-bab45ac0052a None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.379717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccb54c-b698-460a-9637-bab45ac0052a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.405192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-34ccb54c-b698-460a-9637-bab45ac0052a None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:53.407095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-34ccb54c-b698-460a-9637-bab45ac0052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:53.417680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1150/2277] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:53.421697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8759-9f4a-45a7-afbb-4ed8aefa42b4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:53.439660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8759-9f4a-45a7-afbb-4ed8aefa42b4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:53.469103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:53.470088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.470634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.471042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.471863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:07:53.472312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:53.477680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:53.498528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1128/2278] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:07:53.499708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccb54c-b698-460a-9637-bab45ac0052a None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:53.510589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccb54c-b698-460a-9637-bab45ac0052a None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:53.515310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:53.515920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.516201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.516484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.517117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:53.522667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:53.641492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1151/2279] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:07:53 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:07:53.645514 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:53.656798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20827211-fd84-4710-a5f4-9304faaec71c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:53.719984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:53.720736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.721117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.721486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.722254 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:53.724264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:53.825025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1129/2280] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:07:53 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:07:53.829196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:53.840360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da819-b50b-4bc7-9fd4-da9c43fb52c9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:53.844976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:53.845659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.845975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.846251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.846898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:53.848462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:53.862575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1152/2281] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:53.863771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:53.874642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d265ef-6f83-44d1-9ac9-a558b6ac8762 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:53.879554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422fc12-bca6-454a-9fce-efdeaa7291c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:53.883058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422fc12-bca6-454a-9fce-efdeaa7291c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.883058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422fc12-bca6-454a-9fce-efdeaa7291c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.883058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422fc12-bca6-454a-9fce-efdeaa7291c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.883058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422fc12-bca6-454a-9fce-efdeaa7291c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:53.883457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422fc12-bca6-454a-9fce-efdeaa7291c2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:53.893819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1130/2282] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:07:53.898334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:53.908491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0641d5f-b666-4c68-ac21-3232381a3b68 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:53.914041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:53.914685 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:53.914961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:53.915231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] PATH_INFO: `/v3/projects/2f6d01eb0baf462bb2629710235ede70/users/19f9861902804b248c392664469192ba/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:53.931790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6d01eb0baf462bb2629710235ede70, user_id=19f9861902804b248c392664469192ba, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:53.934665 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:54.032467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1153/2283] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:07:53 2022] PUT /identity/v3/projects/2f6d01eb0baf462bb2629710235ede70/users/19f9861902804b248c392664469192ba/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:07:54.043557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064eb09c-b5b5-4ef2-b493-aedc34a8050b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:54.043839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064eb09c-b5b5-4ef2-b493-aedc34a8050b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:54.044110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064eb09c-b5b5-4ef2-b493-aedc34a8050b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:54.075980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-064eb09c-b5b5-4ef2-b493-aedc34a8050b None None] MFA Rules not processed for user `19f9861902804b248c392664469192ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:54.077768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-064eb09c-b5b5-4ef2-b493-aedc34a8050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:54.128868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:54.149155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1131/2284] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:07:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:54.160558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:54.162670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:54.222941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba698b8-0f28-4f0e-bca3-a6e22ddae830 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:54.227507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04084ffe1b0548fea75ea144977e4f15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:54.228107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:54.228389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:54.228663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:54.235005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:07:54.237380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:54.238821 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:54.247917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1154/2285] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:07:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:07:57.008585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad343-497a-433a-a024-5f6569306b96 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:57.009072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad343-497a-433a-a024-5f6569306b96 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:57.009163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad343-497a-433a-a024-5f6569306b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:57.029006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-410ad343-497a-433a-a024-5f6569306b96 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:07:57.030516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-410ad343-497a-433a-a024-5f6569306b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:57.087614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1132/2286] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:07:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:07:57.099915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:07:57.101467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:07:57.173692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae7bf-4260-4e50-a91e-723bc3ce4cc3 service glance] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:07:57.179057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ebfc8-5589-4af4-93f5-2fe5fe63b95c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:07:57.179678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ebfc8-5589-4af4-93f5-2fe5fe63b95c service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:07:57.180006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ebfc8-5589-4af4-93f5-2fe5fe63b95c service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:07:57.180275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ebfc8-5589-4af4-93f5-2fe5fe63b95c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:07:57.189082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ebfc8-5589-4af4-93f5-2fe5fe63b95c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:07:57.190686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ebfc8-5589-4af4-93f5-2fe5fe63b95c service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:07:57.198746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1155/2287] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:07:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:08:07.080609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=31 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:07.196015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=34 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:11.563945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c9973-3116-4dc9-b28c-e250f920badf None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:11.564406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c9973-3116-4dc9-b28c-e250f920badf None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:11.564520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c9973-3116-4dc9-b28c-e250f920badf None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:11.565859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1133/2288] 10.222.0.72 () {54 vars in 961 bytes} [Tue Sep 13 13:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:08:11.571359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:11.571810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:11.571907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:11.591518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:08:11.592702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:11.654668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1156/2289] 10.222.0.72 () {58 vars in 1058 bytes} [Tue Sep 13 13:08:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:08:11.662257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c9973-3116-4dc9-b28c-e250f920badf None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:11.675793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c9973-3116-4dc9-b28c-e250f920badf None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:11.681309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:11.681927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:11.682212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:11.682487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:11.684190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:11.695937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:11.697600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:11.707502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1134/2290] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:08:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:08:21.645224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=29 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:21.703206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=6 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:37.838174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:37.840780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:37.944768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e81fa7-2f2a-4f4b-9afb-887ed481a740 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:37.949184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8326d7-1587-4f8b-b86b-b403776da258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:37.949926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8326d7-1587-4f8b-b86b-b403776da258 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:37.950151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8326d7-1587-4f8b-b86b-b403776da258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:37.950460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8326d7-1587-4f8b-b86b-b403776da258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:37.952750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ab8326d7-1587-4f8b-b86b-b403776da258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:38.045103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8326d7-1587-4f8b-b86b-b403776da258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:38.046987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8326d7-1587-4f8b-b86b-b403776da258 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:38.074156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1157/2291] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:08:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:08:40.770922 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:40.884737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7e175-8b50-438a-a838-dfc00d7c86dd service placement] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:40.889485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:40.890151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:40.890513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:40.890769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] PATH_INFO: `/v3/users/19f9861902804b248c392664469192ba` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:40.891919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f9861902804b248c392664469192ba)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:40.893919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:41.118586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f9861902804b248c392664469192ba'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:41.119461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f9861902804b248c392664469192ba'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:41.128534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f9861902804b248c392664469192ba'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:41.143916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f9861902804b248c392664469192ba'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:41.152118 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f9861902804b248c392664469192ba'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:41.265808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1135/2292] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:08:40 2022] DELETE /identity/v3/users/19f9861902804b248c392664469192ba => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:08:41.269929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:41.270866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:41.271419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:41.295181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:08:41.296802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:41.402052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1158/2293] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:08:41.412349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:41.413828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:41.484495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296c5e5-6c60-4ce6-9d7c-1fbf47df2c07 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:41.488724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db771cc-1db4-416d-ba7e-dcd95a1aa04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:41.489498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db771cc-1db4-416d-ba7e-dcd95a1aa04c service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:41.489861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db771cc-1db4-416d-ba7e-dcd95a1aa04c service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:41.490082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db771cc-1db4-416d-ba7e-dcd95a1aa04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:41.500935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db771cc-1db4-416d-ba7e-dcd95a1aa04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:41.502780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db771cc-1db4-416d-ba7e-dcd95a1aa04c service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:41.514852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1136/2294] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:08:42.008252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:42.009802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:42.030583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af6dcb-f8a3-4ead-bbb6-e879f9a49edc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:42.043079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:42.044518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.045531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.045659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] PATH_INFO: `/v3/projects/2f6d01eb0baf462bb2629710235ede70` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.053644 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6d01eb0baf462bb2629710235ede70)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:42.057032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:42.172709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:42.175661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6d01eb0baf462bb2629710235ede70', 'request_id': 'req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:42.240752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6d01eb0baf462bb2629710235ede70', 'request_id': 'req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:42.248471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6d01eb0baf462bb2629710235ede70', 'request_id': 'req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:42.259329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] The token cache is being invalidate because project 2f6d01eb0baf462bb2629710235ede70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:08:42.259642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:42.260276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-dded5340-e98a-41e2-a8ac-c2f1d8ea73c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:42.261772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1159/2295] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:08:42 2022] DELETE /identity/v3/projects/2f6d01eb0baf462bb2629710235ede70 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:08:42.266615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcef0-d584-439c-b28f-03dc8e702af4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.267084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcef0-d584-439c-b28f-03dc8e702af4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.267198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bcef0-d584-439c-b28f-03dc8e702af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.286498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ba7bcef0-d584-439c-b28f-03dc8e702af4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:08:42.287696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ba7bcef0-d584-439c-b28f-03dc8e702af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:42.305631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:42.348192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1137/2296] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:08:42.357483 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa40813-d05d-4bcd-8623-349e9e9f92df None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.357978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa40813-d05d-4bcd-8623-349e9e9f92df None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.358071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa40813-d05d-4bcd-8623-349e9e9f92df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.378195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5fa40813-d05d-4bcd-8623-349e9e9f92df None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:08:42.379388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5fa40813-d05d-4bcd-8623-349e9e9f92df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:42.387495 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1160/2297] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:08:42.391702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bcef0-d584-439c-b28f-03dc8e702af4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:42.401958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bcef0-d584-439c-b28f-03dc8e702af4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:42.406482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:42.407074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.409678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.409678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.409678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:08:42.409678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:42.410200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:42.422515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1138/2298] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:08:42.426708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa40813-d05d-4bcd-8623-349e9e9f92df None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:42.436245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa40813-d05d-4bcd-8623-349e9e9f92df None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:42.440990 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:42.441589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.441907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.442186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.442824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:42.444692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:42.533902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1161/2299] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:08:42 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:08:42.537876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:42.548818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae2c5c-0898-4e7d-aa2c-bc2c32a937d5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:42.604551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:42.605156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.605432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.605700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.606398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:42.608237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:42.699640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1139/2300] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:08:42 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:08:42.703839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:42.715441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed448ca-092d-4b5b-9c04-092fcc7840d0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:42.722574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:42.723222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.723222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.723222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.723894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:42.725700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:42.737460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1162/2301] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:08:42.741503 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:42.752647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef41b58-d798-47aa-a656-98a1e94f05a9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:42.758139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5616e3-8ea6-4061-8044-e2a4c8cc16b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:42.758750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5616e3-8ea6-4061-8044-e2a4c8cc16b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.759187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5616e3-8ea6-4061-8044-e2a4c8cc16b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.759457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5616e3-8ea6-4061-8044-e2a4c8cc16b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.760145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5616e3-8ea6-4061-8044-e2a4c8cc16b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:42.761879 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5616e3-8ea6-4061-8044-e2a4c8cc16b2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:42.774443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1140/2302] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:08:42.784078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:42.794290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163fc5d-682b-48a1-a12a-8fb8e707ca7e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:42.798913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:42.799563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.799843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.800119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] PATH_INFO: `/v3/projects/86747569a3b84165a4d7c12d5c5451b3/users/c951ee42390947f3b42752e2f658174e/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.815409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86747569a3b84165a4d7c12d5c5451b3, user_id=c951ee42390947f3b42752e2f658174e, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:42.818095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:42.886528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1163/2303] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:08:42 2022] PUT /identity/v3/projects/86747569a3b84165a4d7c12d5c5451b3/users/c951ee42390947f3b42752e2f658174e/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:08:42.896698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217fc4-9bae-42d7-b13e-5754c84680f3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:42.896981 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217fc4-9bae-42d7-b13e-5754c84680f3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:42.897251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217fc4-9bae-42d7-b13e-5754c84680f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:42.924732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-91217fc4-9bae-42d7-b13e-5754c84680f3 None None] MFA Rules not processed for user `c951ee42390947f3b42752e2f658174e`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:08:42.925976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-91217fc4-9bae-42d7-b13e-5754c84680f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:42.989403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1141/2304] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:08:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:08:42.999971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:43.001670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:43.061571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70eea0-c276-4093-884d-c7b85a4da574 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:43.065157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:43.065794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:43.066082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:43.066362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:43.075617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:43.077039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:43.085727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1164/2305] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:08:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:08:43.206275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bcc4f-641d-4840-8d18-7eb87817cf87 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:43.206275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bcc4f-641d-4840-8d18-7eb87817cf87 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:43.206275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bcc4f-641d-4840-8d18-7eb87817cf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:43.227683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-082bcc4f-641d-4840-8d18-7eb87817cf87 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:08:43.228893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-082bcc4f-641d-4840-8d18-7eb87817cf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:43.244892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:43.292813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1142/2306] 10.222.0.72 () {58 vars in 967 bytes} [Tue Sep 13 13:08:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:08:43.317412 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:43.318928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:43.389395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050684a-1683-4843-b908-57c32e269ab3 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:43.395580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69325810-f241-4b5d-af03-d08529c04d78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:43.396184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325810-f241-4b5d-af03-d08529c04d78 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:43.396469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325810-f241-4b5d-af03-d08529c04d78 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:43.396730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325810-f241-4b5d-af03-d08529c04d78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:43.405019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69325810-f241-4b5d-af03-d08529c04d78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:43.406725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69325810-f241-4b5d-af03-d08529c04d78 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:43.415051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1165/2307] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:08:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:08:53.303895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=17 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:53.411586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:08:54.367288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bcc4f-641d-4840-8d18-7eb87817cf87 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:54.430229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bcc4f-641d-4840-8d18-7eb87817cf87 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:54.437277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:54.437938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:54.438224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:54.438490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] PATH_INFO: `/v3/users/c951ee42390947f3b42752e2f658174e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:54.439612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c951ee42390947f3b42752e2f658174e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:54.441520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:54.620082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c951ee42390947f3b42752e2f658174e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:54.620718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c951ee42390947f3b42752e2f658174e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:54.627779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c951ee42390947f3b42752e2f658174e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:54.641226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c951ee42390947f3b42752e2f658174e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:54.641998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c951ee42390947f3b42752e2f658174e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:54.698346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1143/2308] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:08:54 2022] DELETE /identity/v3/users/c951ee42390947f3b42752e2f658174e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:08:54.707715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:54.708127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:54.708269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:54.728934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:08:54.730181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:54.789447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1166/2309] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:08:54.799446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:54.800848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:54.871690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1f0d3-b18a-4431-835c-f41be9a99cff admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:54.875868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a1bf8-f2b6-4c71-b93e-15cdb1e6e13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:54.876672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1bf8-f2b6-4c71-b93e-15cdb1e6e13a service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:54.876955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1bf8-f2b6-4c71-b93e-15cdb1e6e13a service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:54.877236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1bf8-f2b6-4c71-b93e-15cdb1e6e13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:54.887859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a1bf8-f2b6-4c71-b93e-15cdb1e6e13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:54.889289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a1bf8-f2b6-4c71-b93e-15cdb1e6e13a service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:54.899417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1144/2310] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:08:55.287880 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:08:55.289332 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:08:55.301382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13734f94-ea7c-429d-967b-f1c431df01ac None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:08:55.306949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:08:55.307586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:08:55.307900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:08:55.308257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] PATH_INFO: `/v3/projects/86747569a3b84165a4d7c12d5c5451b3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:08:55.309372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86747569a3b84165a4d7c12d5c5451b3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:08:55.311305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:08:55.475343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86747569a3b84165a4d7c12d5c5451b3', 'request_id': 'req-06fc2f90-7203-4c08-a563-78d974ec9957'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:55.548411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86747569a3b84165a4d7c12d5c5451b3', 'request_id': 'req-06fc2f90-7203-4c08-a563-78d974ec9957'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:55.556369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86747569a3b84165a4d7c12d5c5451b3', 'request_id': 'req-06fc2f90-7203-4c08-a563-78d974ec9957'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:55.566919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] The token cache is being invalidate because project 86747569a3b84165a4d7c12d5c5451b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:08:55.567213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:55.567855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06fc2f90-7203-4c08-a563-78d974ec9957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:08:55.569368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1167/2311] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:08:55 2022] DELETE /identity/v3/projects/86747569a3b84165a4d7c12d5c5451b3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:09:04.894737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:09:05.563670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:09:06.676464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32f04b-7dbf-4c91-b991-571419a2f959 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:09:06.676994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32f04b-7dbf-4c91-b991-571419a2f959 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:09:06.676994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32f04b-7dbf-4c91-b991-571419a2f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:09:06.697750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0a32f04b-7dbf-4c91-b991-571419a2f959 None None] MFA Rules not processed for user `9fadb80f21614816b3526d59b2d8a0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:09:06.698966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0a32f04b-7dbf-4c91-b991-571419a2f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:09:06.755829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1145/2312] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:09:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:09:11.997543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:09:11.998080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:09:12.003860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:09:12.019423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] MFA Rules not processed for user `8c688fff323943b3bbcbb0569d974d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:09:12.020652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:09:12.077287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1168/2313] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:09:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:09:16.748356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=25 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:09:22.070307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=25 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:09:36.098166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f086f-385c-4614-b1f4-c8bb74c7a19d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:09:36.098774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f086f-385c-4614-b1f4-c8bb74c7a19d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:09:36.098774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f086f-385c-4614-b1f4-c8bb74c7a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:09:36.117706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-d34f086f-385c-4614-b1f4-c8bb74c7a19d None None] MFA Rules not processed for user `fe3be97820d64f85aa4d2dbd8f3ffd33`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:09:36.118911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d34f086f-385c-4614-b1f4-c8bb74c7a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:09:36.172727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1146/2314] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:09:46.166143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=25 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:09:56.232301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:09:56.233758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:09:56.293003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82b01-df63-48d9-96d4-b9f78f637903 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:09:56.296594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c1002-846b-4344-8889-3a62bc883320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:09:56.297222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c1002-846b-4344-8889-3a62bc883320 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:09:56.297500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c1002-846b-4344-8889-3a62bc883320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:09:56.297801 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c1002-846b-4344-8889-3a62bc883320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:09:56.299552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-077c1002-846b-4344-8889-3a62bc883320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:09:56.370320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c1002-846b-4344-8889-3a62bc883320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:09:56.371741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c1002-846b-4344-8889-3a62bc883320 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:09:56.382728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1169/2315] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:09:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:09:57.562614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f086f-385c-4614-b1f4-c8bb74c7a19d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:09:57.572394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f086f-385c-4614-b1f4-c8bb74c7a19d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:09:57.575882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:09:57.576472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:09:57.576745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:09:57.577012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:09:57.578767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:09:57.592392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:09:57.593840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:09:57.605064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1147/2316] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:09:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:04.224020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:04.224604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:04.224604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:04.245466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] MFA Rules not processed for user `7b1c15ff53e5442881b56bf504b9a31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:04.248551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:04.311056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1170/2317] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:04.894049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:04.903174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ae1b-ad3b-4c16-a693-d55bc3556611 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:04.906835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:04.907872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:04.907872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:04.908112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:04.909691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:04.919509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:04.920958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:04.929196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1148/2318] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:05.894155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:05.895580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:05.950461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:05.972499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f49e5-4b6d-4042-b7b6-21f2b7a641b3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:05.978085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:05.978713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:05.978998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:05.979274 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:05.981084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:05.994235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:05.995721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:06.010564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1171/2319] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:10.265391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:10.267089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:10.285633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a3e48-356f-4248-ba43-050bd1f3f6bc service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:10.289790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:10.290369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:10.290642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:10.290911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:10.293417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:10.303754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:10.305302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:10.314456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1149/2320] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:16.002443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=11 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:17.543492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:17.553082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83277ee-1140-42ae-888b-9b6d33d794ff service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:17.556625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:17.557290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:17.557546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:17.558447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:17.560172 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:17.574770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:17.574770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:17.579356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1172/2321] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:20.310141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=18 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:22.349020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:22.359035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18be626-0261-4991-8bf8-ec24314aed2b service nova] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:22.362633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:22.363225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:22.363501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:22.363769 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:22.365419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:22.375025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:22.376428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:22.377283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:22.378803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:22.388648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1150/2322] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:22.396349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad5c15-cfb6-4647-8d6d-5262ad62089c service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:22.402738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:22.403439 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:22.403781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:22.404123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:22.405974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:22.416980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:22.418643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:22.427760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1173/2323] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:10:23.138142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:23.150822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac6c7d-838f-47d5-8f03-73f5979df21f service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:23.158274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:23.158887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:23.159163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:23.159433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:23.162196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:23.180768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:23.182438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:23.194775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1151/2324] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:24.339874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:24.339874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:24.349633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07902d74-e50f-4921-8810-fe39ca062823 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:24.353408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4df36a-887c-4b1b-9a82-162512d84ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:24.354132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df36a-887c-4b1b-9a82-162512d84ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:24.354482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df36a-887c-4b1b-9a82-162512d84ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:24.354863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df36a-887c-4b1b-9a82-162512d84ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:24.356906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6b4df36a-887c-4b1b-9a82-162512d84ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:24.473428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4df36a-887c-4b1b-9a82-162512d84ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:24.475057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4df36a-887c-4b1b-9a82-162512d84ef9 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:24.486721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1174/2325] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:27.307775 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:27.320317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7cd63-9adc-447d-ad2c-2ad64f5fb206 service nova] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:27.327598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70adb0d-f391-4b20-8af6-b410d5ece6cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:27.328534 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70adb0d-f391-4b20-8af6-b410d5ece6cc service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:27.328965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70adb0d-f391-4b20-8af6-b410d5ece6cc service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:27.329510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70adb0d-f391-4b20-8af6-b410d5ece6cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:27.331597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d70adb0d-f391-4b20-8af6-b410d5ece6cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:27.343980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70adb0d-f391-4b20-8af6-b410d5ece6cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:27.345676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70adb0d-f391-4b20-8af6-b410d5ece6cc service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:27.356956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1152/2326] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:28.172484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266e8f4-3859-428f-bc5e-7f9c578d2b6d None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:28.172995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266e8f4-3859-428f-bc5e-7f9c578d2b6d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:28.173115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266e8f4-3859-428f-bc5e-7f9c578d2b6d None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:28.174377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1175/2327] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:10:28.185127 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:28.193136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:28.193136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:28.206891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] MFA Rules not processed for user `dca4df9896df47268d091c3b32cbe8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:28.208347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:28.221293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1153/2328] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:10:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:28.234471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266e8f4-3859-428f-bc5e-7f9c578d2b6d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:28.261213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266e8f4-3859-428f-bc5e-7f9c578d2b6d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:28.261213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:28.261213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:28.261213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:28.261695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:28.271369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:28.273017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:28.284279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1176/2329] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:28.316380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:28.317829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:28.334896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797adfaa-9b62-41a6-99ba-27e6596c21d3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:28.340034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:28.340812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:28.341330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:28.341649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] PATH_INFO: `/v3/users/8c688fff323943b3bbcbb0569d974d0a` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:28.342984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c688fff323943b3bbcbb0569d974d0a)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:28.345010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:28.479223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c688fff323943b3bbcbb0569d974d0a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:28.480209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c688fff323943b3bbcbb0569d974d0a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:28.487859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c688fff323943b3bbcbb0569d974d0a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:28.500855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c688fff323943b3bbcbb0569d974d0a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:28.501575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-37e447bb-97e0-4bcb-970d-9504fd74f3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c688fff323943b3bbcbb0569d974d0a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:28.609958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1154/2330] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:10:28 2022] DELETE /identity/v3/users/8c688fff323943b3bbcbb0569d974d0a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:29.009113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:29.023573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe75ec-9fe1-41e5-ac08-b6786f326416 service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:29.099444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:29.100268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:29.100618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:29.100959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] PATH_INFO: `/v3/projects/c15624f4312f49bb85a7a086c291d157` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:29.102204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15624f4312f49bb85a7a086c291d157)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:29.104386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:29.285119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15624f4312f49bb85a7a086c291d157', 'request_id': 'req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:29.346676 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15624f4312f49bb85a7a086c291d157', 'request_id': 'req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:29.348192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:29.354812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15624f4312f49bb85a7a086c291d157', 'request_id': 'req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:29.365701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] The token cache is being invalidate because project c15624f4312f49bb85a7a086c291d157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:10:29.366016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:29.366674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2419b52f-f9cd-4be6-a9d6-d9ed80d07415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:29.368641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1177/2331] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:10:29 2022] DELETE /identity/v3/projects/c15624f4312f49bb85a7a086c291d157 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:29.374621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc22d6-c1e1-4d48-8522-12feaf342ff4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:29.375117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc22d6-c1e1-4d48-8522-12feaf342ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:29.375217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc22d6-c1e1-4d48-8522-12feaf342ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:29.396638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-89bc22d6-c1e1-4d48-8522-12feaf342ff4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:29.397877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-89bc22d6-c1e1-4d48-8522-12feaf342ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:29.472598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1155/2332] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:29.480982 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220ac5-f89f-45c2-8da8-4cbb48b0965e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:29.481334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220ac5-f89f-45c2-8da8-4cbb48b0965e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:29.481633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220ac5-f89f-45c2-8da8-4cbb48b0965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:29.502965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-0e220ac5-f89f-45c2-8da8-4cbb48b0965e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:29.504722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-0e220ac5-f89f-45c2-8da8-4cbb48b0965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:29.521869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1178/2333] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:29.522333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc22d6-c1e1-4d48-8522-12feaf342ff4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:29.532137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc22d6-c1e1-4d48-8522-12feaf342ff4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:29.539080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:29.539778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:29.540570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:29.543283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:29.543283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:10:29.543283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:29.544338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:29.556839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1156/2334] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:10:29.562185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220ac5-f89f-45c2-8da8-4cbb48b0965e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:29.574208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220ac5-f89f-45c2-8da8-4cbb48b0965e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:29.580425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:29.581064 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:29.581343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:29.581643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:29.582316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:29.584296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:29.785766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1179/2335] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:10:29 2022] POST /identity/v3/projects => generated 403 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:29.797234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:29.798724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb63f8-eaf4-43ed-abcc-bde7143573fc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:29.808705 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:29.857351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:29.858093 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:29.858367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:29.858639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:29.859296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:29.861236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:29.969610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1157/2336] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:10:29 2022] POST /identity/v3/users => generated 302 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:29.970042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:29.983682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660fa4e-eb4a-4ca8-8439-6858ac775377 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:29.989988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:29.990497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:29.990771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:29.991043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:29.991725 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:29.993380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:30.021498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1180/2337] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:10:30.022359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:30.035605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6333d4d-c1a8-4426-8bd7-21d91f6a8d98 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:30.039535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468b09d-58b7-406f-abd7-cdf50b8ce57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:30.040154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468b09d-58b7-406f-abd7-cdf50b8ce57f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:30.040432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468b09d-58b7-406f-abd7-cdf50b8ce57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:30.040700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468b09d-58b7-406f-abd7-cdf50b8ce57f admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:30.041352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468b09d-58b7-406f-abd7-cdf50b8ce57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:30.043078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468b09d-58b7-406f-abd7-cdf50b8ce57f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:30.054899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1158/2338] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:30.060011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:30.070811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f702d-06b8-4864-84c8-0e4ebf4299a0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:30.078701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:30.079494 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:30.079958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:30.080311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] PATH_INFO: `/v3/projects/4caa315bd06c4d3e94c497abb4ea3857/users/127c8cff5c3e4dbdb7c0b669928b03af/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:30.096260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4caa315bd06c4d3e94c497abb4ea3857, user_id=127c8cff5c3e4dbdb7c0b669928b03af, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:30.099152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:30.212443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1181/2339] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:10:30 2022] PUT /identity/v3/projects/4caa315bd06c4d3e94c497abb4ea3857/users/127c8cff5c3e4dbdb7c0b669928b03af/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:10:30.218773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359aa1-b5c7-490c-8c63-c2796a7cfe5d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:30.219210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359aa1-b5c7-490c-8c63-c2796a7cfe5d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:30.219553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359aa1-b5c7-490c-8c63-c2796a7cfe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:30.247616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-8a359aa1-b5c7-490c-8c63-c2796a7cfe5d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:30.248841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8a359aa1-b5c7-490c-8c63-c2796a7cfe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:30.322954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1159/2340] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:30.332729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:30.334957 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:30.394181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcabb5-8f44-4d49-ba42-0c5f4db42647 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:30.397934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:30.398535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:30.398812 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:30.399080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:30.408393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:30.409894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:30.420534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1182/2341] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:31.753840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548b138-2f9b-405e-8826-008bb61381e8 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:31.754382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548b138-2f9b-405e-8826-008bb61381e8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:31.754382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548b138-2f9b-405e-8826-008bb61381e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:31.785638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2548b138-2f9b-405e-8826-008bb61381e8 None None] MFA Rules not processed for user `127c8cff5c3e4dbdb7c0b669928b03af`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:31.787575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2548b138-2f9b-405e-8826-008bb61381e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:31.856402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1160/2342] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:31.861046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:31.871303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b94b1a-efa6-446d-9517-a224875916b6 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:31.875852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:31.876448 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:31.876729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:31.877003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:31.877774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:31.879599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:31.978819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1183/2343] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:10:31 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:31.987407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548b138-2f9b-405e-8826-008bb61381e8 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:31.998375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548b138-2f9b-405e-8826-008bb61381e8 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:32.017357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:32.062226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:32.062876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:32.063158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:32.063434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:32.064074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:32.066192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:32.186903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1161/2344] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:10:31 2022] POST /identity/v3/users => generated 309 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:10:32.193246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:32.203705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d556f-eb08-46a8-8546-302b6df792ee admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:32.208618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:32.219099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:32.219099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:32.219099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:32.219099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:32.219099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:32.223726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1184/2345] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:32.228088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:32.238668 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232e401-537a-4bd2-87ca-38f108c65752 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:32.248894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:32.248894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:32.248894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:32.248894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] PATH_INFO: `/v3/projects/256b27ccf0a94565af3a5091e4986ed0/users/59987467b12b4b3ab0aaca650cdc6da8/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:32.259485 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256b27ccf0a94565af3a5091e4986ed0, user_id=59987467b12b4b3ab0aaca650cdc6da8, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:32.262213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:32.341412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1162/2346] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:10:32 2022] PUT /identity/v3/projects/256b27ccf0a94565af3a5091e4986ed0/users/59987467b12b4b3ab0aaca650cdc6da8/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:32.345403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:32.356839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643685a-bd35-4d9b-85a0-ea8f12b0670e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:32.403220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:32.413341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d1807-a78c-4dd1-b6ca-aa3d7c893063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:32.413992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1807-a78c-4dd1-b6ca-aa3d7c893063 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:32.414282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1807-a78c-4dd1-b6ca-aa3d7c893063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:32.414557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1807-a78c-4dd1-b6ca-aa3d7c893063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:32.415202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d1807-a78c-4dd1-b6ca-aa3d7c893063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:32.416826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d1807-a78c-4dd1-b6ca-aa3d7c893063 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:32.430388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1185/2347] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:10:32.435280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:32.445617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed1cfa-792f-4175-b7a3-00bb48b72b23 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:32.452531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49dd4-3935-4a8f-b1cc-7293a8cd97ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:32.452531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49dd4-3935-4a8f-b1cc-7293a8cd97ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:32.452531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49dd4-3935-4a8f-b1cc-7293a8cd97ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:32.452531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49dd4-3935-4a8f-b1cc-7293a8cd97ea admin admin] PATH_INFO: `/v3/projects/256b27ccf0a94565af3a5091e4986ed0/users/59987467b12b4b3ab0aaca650cdc6da8/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:32.453733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49dd4-3935-4a8f-b1cc-7293a8cd97ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=256b27ccf0a94565af3a5091e4986ed0, user_id=59987467b12b4b3ab0aaca650cdc6da8, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:32.456396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49dd4-3935-4a8f-b1cc-7293a8cd97ea admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:32.466891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1163/2348] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:10:32 2022] PUT /identity/v3/projects/256b27ccf0a94565af3a5091e4986ed0/users/59987467b12b4b3ab0aaca650cdc6da8/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:10:33.926509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0a522-67da-4a14-b8ad-8064d21ca31c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:33.928191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0a522-67da-4a14-b8ad-8064d21ca31c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:33.928191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0a522-67da-4a14-b8ad-8064d21ca31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:33.959723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-3db0a522-67da-4a14-b8ad-8064d21ca31c None None] MFA Rules not processed for user `59987467b12b4b3ab0aaca650cdc6da8`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:33.961019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-3db0a522-67da-4a14-b8ad-8064d21ca31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:34.037313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1186/2349] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:34.224040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951d0fd-2f8f-443e-a4cf-634b9e016894 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:34.224592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951d0fd-2f8f-443e-a4cf-634b9e016894 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:34.224592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951d0fd-2f8f-443e-a4cf-634b9e016894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:34.247783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7951d0fd-2f8f-443e-a4cf-634b9e016894 None None] MFA Rules not processed for user `9fadb80f21614816b3526d59b2d8a0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:34.249016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7951d0fd-2f8f-443e-a4cf-634b9e016894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:34.305409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1164/2350] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:10:35.684393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0a522-67da-4a14-b8ad-8064d21ca31c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:35.753498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0a522-67da-4a14-b8ad-8064d21ca31c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:35.758349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:35.758986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:35.759324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:35.759602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] PATH_INFO: `/v3/users/127c8cff5c3e4dbdb7c0b669928b03af` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:35.760780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127c8cff5c3e4dbdb7c0b669928b03af)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:35.762795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:35.983704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127c8cff5c3e4dbdb7c0b669928b03af'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:35.984476 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127c8cff5c3e4dbdb7c0b669928b03af'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:35.991936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127c8cff5c3e4dbdb7c0b669928b03af'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.004586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127c8cff5c3e4dbdb7c0b669928b03af'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.005225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127c8cff5c3e4dbdb7c0b669928b03af'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.006997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:36.090565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1187/2351] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:10:35 2022] DELETE /identity/v3/users/127c8cff5c3e4dbdb7c0b669928b03af => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:36.350072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951d0fd-2f8f-443e-a4cf-634b9e016894 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:36.362336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951d0fd-2f8f-443e-a4cf-634b9e016894 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:36.421193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:36.421876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:36.422157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:36.422432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] PATH_INFO: `/v3/projects/4caa315bd06c4d3e94c497abb4ea3857` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:36.423581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4caa315bd06c4d3e94c497abb4ea3857)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:36.425493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:36.449242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:36.553325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4caa315bd06c4d3e94c497abb4ea3857', 'request_id': 'req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.631633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4caa315bd06c4d3e94c497abb4ea3857', 'request_id': 'req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.639516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4caa315bd06c4d3e94c497abb4ea3857', 'request_id': 'req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.650235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] The token cache is being invalidate because project 4caa315bd06c4d3e94c497abb4ea3857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:10:36.650527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.651278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.657777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1165/2352] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:10:36 2022] DELETE /identity/v3/projects/4caa315bd06c4d3e94c497abb4ea3857 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:36.662221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:36.663773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:36.677545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048956-109f-4613-8667-85c43c5b82f9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:36.734207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:36.734907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:36.735108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:36.735382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] PATH_INFO: `/v3/users/59987467b12b4b3ab0aaca650cdc6da8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:36.736498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59987467b12b4b3ab0aaca650cdc6da8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:36.738434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:36.899063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59987467b12b4b3ab0aaca650cdc6da8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.899269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59987467b12b4b3ab0aaca650cdc6da8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.906655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59987467b12b4b3ab0aaca650cdc6da8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.919943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59987467b12b4b3ab0aaca650cdc6da8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:36.920722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f95c1b10-5743-4fc3-a27f-25988087a143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59987467b12b4b3ab0aaca650cdc6da8'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:37.025666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1188/2353] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:10:36 2022] DELETE /identity/v3/users/59987467b12b4b3ab0aaca650cdc6da8 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:37.252132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:37.253484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:37.267885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c38dde-6d5f-45ef-8d63-5cf0f89544a0 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:37.331121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:37.331906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:37.332185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:37.332461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] PATH_INFO: `/v3/projects/256b27ccf0a94565af3a5091e4986ed0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:37.333641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=256b27ccf0a94565af3a5091e4986ed0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:37.335726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:37.462761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256b27ccf0a94565af3a5091e4986ed0', 'request_id': 'req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:37.534865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256b27ccf0a94565af3a5091e4986ed0', 'request_id': 'req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:37.543368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256b27ccf0a94565af3a5091e4986ed0', 'request_id': 'req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:37.556128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] The token cache is being invalidate because project 256b27ccf0a94565af3a5091e4986ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:10:37.556466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:37.557274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-3c72ac3c-bceb-4b9a-af18-d7abe0d03847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:37.559056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1166/2354] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:10:37 2022] DELETE /identity/v3/projects/256b27ccf0a94565af3a5091e4986ed0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:37.564213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711980d7-ac7b-4056-8954-db0784841028 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:37.566129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711980d7-ac7b-4056-8954-db0784841028 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:37.566129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711980d7-ac7b-4056-8954-db0784841028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:37.587014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-711980d7-ac7b-4056-8954-db0784841028 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:37.588381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-711980d7-ac7b-4056-8954-db0784841028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:37.674713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1189/2355] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:37.683592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4c541-728b-4d31-926a-d2a7c2e559eb None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:37.684091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4c541-728b-4d31-926a-d2a7c2e559eb None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:37.684551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4c541-728b-4d31-926a-d2a7c2e559eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:37.708670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-84d4c541-728b-4d31-926a-d2a7c2e559eb None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:37.710034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-84d4c541-728b-4d31-926a-d2a7c2e559eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:37.718806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1167/2356] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:37.723187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711980d7-ac7b-4056-8954-db0784841028 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:37.733008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711980d7-ac7b-4056-8954-db0784841028 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:37.738100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:37.740346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:37.740346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:37.740346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:37.740346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:10:37.740346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:37.741818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:37.749197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:37.757852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1190/2357] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:10:37.759365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4c541-728b-4d31-926a-d2a7c2e559eb None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:37.770217 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4c541-728b-4d31-926a-d2a7c2e559eb None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:37.780697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:37.780697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:37.780697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:37.780697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:37.781237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:37.781772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:37.867665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1168/2358] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:10:37 2022] POST /identity/v3/projects => generated 409 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:37.872165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:37.882000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee559bf-f60c-4002-a888-e9c7f0fc65e4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:37.942745 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:37.943357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:37.943666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:37.943941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:37.944585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:37.946521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.033008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1191/2359] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:10:37 2022] POST /identity/v3/users => generated 305 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:38.037543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.048281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823e3a2-2c2a-4d79-9951-c4b203ceb633 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:38.052983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:38.055946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.055946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.055946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.055946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:38.056363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.062590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:38.067684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1169/2360] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:38.071819 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.082481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b64c-af30-42d5-93d9-2a98b8a0364a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:38.087133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef686ac-40b4-48f2-a81f-af6905efe21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:38.087734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef686ac-40b4-48f2-a81f-af6905efe21b admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.088007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef686ac-40b4-48f2-a81f-af6905efe21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.088279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef686ac-40b4-48f2-a81f-af6905efe21b admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.089139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef686ac-40b4-48f2-a81f-af6905efe21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:38.090751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef686ac-40b4-48f2-a81f-af6905efe21b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.101435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1192/2361] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:38.105708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.116405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2610ed9-c6a5-4740-a733-9e8bcc953642 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:38.122715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:38.125306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.125306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.125306 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] PATH_INFO: `/v3/projects/cc6ed2e595254764b74919e1a484863d/users/f49aa0d45b0c4f45bc9e378e8ab9b1fa/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.141014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6ed2e595254764b74919e1a484863d, user_id=f49aa0d45b0c4f45bc9e378e8ab9b1fa, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:38.148391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.231334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1170/2362] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:10:38 2022] PUT /identity/v3/projects/cc6ed2e595254764b74919e1a484863d/users/f49aa0d45b0c4f45bc9e378e8ab9b1fa/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:38.232816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd368b5-f0a6-4e65-8f72-dc761623740d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.233124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd368b5-f0a6-4e65-8f72-dc761623740d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.233427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd368b5-f0a6-4e65-8f72-dc761623740d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.267307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ebd368b5-f0a6-4e65-8f72-dc761623740d None None] MFA Rules not processed for user `f49aa0d45b0c4f45bc9e378e8ab9b1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:38.268842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ebd368b5-f0a6-4e65-8f72-dc761623740d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:38.351766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1193/2363] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:38 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:38.356973 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.429689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b6596-b8d0-4c2e-865b-73f7c57dfb74 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:38.435750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:38.436794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.437353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.438061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.438758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:38.440664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.568445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1171/2364] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:10:38 2022] POST /identity/v3/projects => generated 407 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:38.569511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd368b5-f0a6-4e65-8f72-dc761623740d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.581413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd368b5-f0a6-4e65-8f72-dc761623740d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:38.644724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:38.648657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.648657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.648657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.648657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:38.649028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.660610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:38.773911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1194/2365] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:10:38 2022] POST /identity/v3/users => generated 310 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:38.778513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.789149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193908c-a7df-4da3-b4ab-f4e594d6a487 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:38.794837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:38.795520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.795900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.796245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.796961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:38.798844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.809791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1172/2366] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:38.821054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.825068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070b8cb-8ae1-489d-b955-050cdc3f8600 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:38.829726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:38.830420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:38.830701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:38.831007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] PATH_INFO: `/v3/projects/0af34b63bd824fac8ff1c2821d9af53e/users/cd5cb935f9b146338bf47e3e1e67dd2c/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:38.847597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af34b63bd824fac8ff1c2821d9af53e, user_id=cd5cb935f9b146338bf47e3e1e67dd2c, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:38.850280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:38.928967 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1195/2367] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:10:38 2022] PUT /identity/v3/projects/0af34b63bd824fac8ff1c2821d9af53e/users/cd5cb935f9b146338bf47e3e1e67dd2c/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:38.933119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:38.943714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e684358-9344-4040-9e91-e688fe1629f4 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:39.004019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd01bdf-4d6b-45bd-88c3-8a5b8e05ea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:39.004628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01bdf-4d6b-45bd-88c3-8a5b8e05ea34 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:39.004910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01bdf-4d6b-45bd-88c3-8a5b8e05ea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:39.005169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd01bdf-4d6b-45bd-88c3-8a5b8e05ea34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:39.005869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd01bdf-4d6b-45bd-88c3-8a5b8e05ea34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:39.007549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd01bdf-4d6b-45bd-88c3-8a5b8e05ea34 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:39.017470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1173/2368] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:39.022441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:39.033482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0ffca-0e67-42e1-95db-b03e0e8efceb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:39.038429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:39.039097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:39.039378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:39.039658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] PATH_INFO: `/v3/projects/0af34b63bd824fac8ff1c2821d9af53e/users/cd5cb935f9b146338bf47e3e1e67dd2c/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:39.041564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af34b63bd824fac8ff1c2821d9af53e, user_id=cd5cb935f9b146338bf47e3e1e67dd2c, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:39.044384 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:39.055054 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1196/2369] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:10:39 2022] PUT /identity/v3/projects/0af34b63bd824fac8ff1c2821d9af53e/users/cd5cb935f9b146338bf47e3e1e67dd2c/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:10:39.065945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a8aa-7ea6-432b-837a-7515b4936e09 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:39.066235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a8aa-7ea6-432b-837a-7515b4936e09 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:39.066512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a8aa-7ea6-432b-837a-7515b4936e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:39.095604 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f711a8aa-7ea6-432b-837a-7515b4936e09 None None] MFA Rules not processed for user `cd5cb935f9b146338bf47e3e1e67dd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:39.096839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f711a8aa-7ea6-432b-837a-7515b4936e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:39.132862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:39.167529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1174/2370] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:39 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:39.178781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:39.180503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:39.241701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dcc70-ff15-421f-bdff-da2557189df6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:39.246128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:39.246598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:39.246880 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:39.247238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:39.257370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:39.259015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:39.270499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1197/2371] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:39 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:39.598416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a8aa-7ea6-432b-837a-7515b4936e09 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:39.608666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a8aa-7ea6-432b-837a-7515b4936e09 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:39.612820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:39.613364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:39.613639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:39.613941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:39.673500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:39.675000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:39.684163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1175/2372] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:39 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:47.272571 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:47.274028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:47.348370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1cbc4-a893-497e-a017-735c023d3970 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:47.354166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc214c-2726-4923-be8c-8d75aa1a9d92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:47.360655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc214c-2726-4923-be8c-8d75aa1a9d92 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:47.360655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc214c-2726-4923-be8c-8d75aa1a9d92 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:47.360655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc214c-2726-4923-be8c-8d75aa1a9d92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:47.360655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a0cc214c-2726-4923-be8c-8d75aa1a9d92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:47.371258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc214c-2726-4923-be8c-8d75aa1a9d92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:47.372905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc214c-2726-4923-be8c-8d75aa1a9d92 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:47.388000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1198/2373] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:49.561045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:49.627048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f021f-7c52-413d-9be2-90307dae95af service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:49.634177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:49.634177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:49.634177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:49.634177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] PATH_INFO: `/v3/users/f49aa0d45b0c4f45bc9e378e8ab9b1fa` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:49.636634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49aa0d45b0c4f45bc9e378e8ab9b1fa)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:49.636771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:49.853186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49aa0d45b0c4f45bc9e378e8ab9b1fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:49.853873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49aa0d45b0c4f45bc9e378e8ab9b1fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:49.862418 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49aa0d45b0c4f45bc9e378e8ab9b1fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:49.875620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:49.883644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49aa0d45b0c4f45bc9e378e8ab9b1fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:49.883644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49aa0d45b0c4f45bc9e378e8ab9b1fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:49.967561 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1176/2374] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:10:49 2022] DELETE /identity/v3/users/f49aa0d45b0c4f45bc9e378e8ab9b1fa => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:10:49.971775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:49.972257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:49.972363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:49.991326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:49.992463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:50.037547 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:50.069987 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1199/2375] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:50.075749 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:50.077183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:50.139867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e760db-02ec-4ba4-93a2-c9fd691e2599 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:50.143517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:50.144131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:50.144413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:50.144778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:50.154312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:50.155750 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:50.165590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1177/2376] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:10:50.819055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:50.819055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:50.837903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad43597-96d5-4d32-8031-a4a96f4abd22 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:50.847942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:50.848541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:50.848826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:50.849101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] PATH_INFO: `/v3/projects/cc6ed2e595254764b74919e1a484863d` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:50.850453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6ed2e595254764b74919e1a484863d)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:50.852944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:50.964917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6ed2e595254764b74919e1a484863d', 'request_id': 'req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.048428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6ed2e595254764b74919e1a484863d', 'request_id': 'req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.055341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6ed2e595254764b74919e1a484863d', 'request_id': 'req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.066489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] The token cache is being invalidate because project cc6ed2e595254764b74919e1a484863d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:10:51.066788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.067519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.069039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1200/2377] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:10:50 2022] DELETE /identity/v3/projects/cc6ed2e595254764b74919e1a484863d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:51.073674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:51.084171 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde563e-3aab-46f6-a3fd-1bf44711d8e1 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:51.154952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:51.155691 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:51.156034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:51.156373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] PATH_INFO: `/v3/users/cd5cb935f9b146338bf47e3e1e67dd2c` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:51.157573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5cb935f9b146338bf47e3e1e67dd2c)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:51.159622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:51.378798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5cb935f9b146338bf47e3e1e67dd2c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.379945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5cb935f9b146338bf47e3e1e67dd2c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.390896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5cb935f9b146338bf47e3e1e67dd2c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.408694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5cb935f9b146338bf47e3e1e67dd2c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.409855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-26af2e44-f8f2-4d47-ae21-58519e4452dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5cb935f9b146338bf47e3e1e67dd2c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:51.523626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1178/2378] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:10:51 2022] DELETE /identity/v3/users/cd5cb935f9b146338bf47e3e1e67dd2c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:10:52.071552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:52.073231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:52.087380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244c5c-4a4d-4a68-9a97-a1ecdec0cf58 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:52.166616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:52.167918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.168216 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.168527 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] PATH_INFO: `/v3/projects/0af34b63bd824fac8ff1c2821d9af53e` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.169697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af34b63bd824fac8ff1c2821d9af53e)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:52.171811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:52.331042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af34b63bd824fac8ff1c2821d9af53e', 'request_id': 'req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:52.386100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af34b63bd824fac8ff1c2821d9af53e', 'request_id': 'req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:52.395225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af34b63bd824fac8ff1c2821d9af53e', 'request_id': 'req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:52.403702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:52.409232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] The token cache is being invalidate because project 0af34b63bd824fac8ff1c2821d9af53e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:10:52.409525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:52.410259 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b6448eb9-58b5-4d53-b04d-13878d8bbe1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:10:52.412325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1201/2379] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:10:52 2022] DELETE /identity/v3/projects/0af34b63bd824fac8ff1c2821d9af53e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:52.417150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4f2af-0257-4e2e-981c-1e4f57e02929 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.418069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4f2af-0257-4e2e-981c-1e4f57e02929 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.418569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4f2af-0257-4e2e-981c-1e4f57e02929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.443410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-43a4f2af-0257-4e2e-981c-1e4f57e02929 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:52.444409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-43a4f2af-0257-4e2e-981c-1e4f57e02929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:52.450890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:52.516580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1179/2380] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:52.524545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce3b5a-52fb-4f64-9ab8-bbe9e0b3c6c8 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.524874 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce3b5a-52fb-4f64-9ab8-bbe9e0b3c6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.525095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce3b5a-52fb-4f64-9ab8-bbe9e0b3c6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.551702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-00ce3b5a-52fb-4f64-9ab8-bbe9e0b3c6c8 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:52.552978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-00ce3b5a-52fb-4f64-9ab8-bbe9e0b3c6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:52.561518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1202/2381] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:52.571350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4f2af-0257-4e2e-981c-1e4f57e02929 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:52.582437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4f2af-0257-4e2e-981c-1e4f57e02929 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:52.587850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:52.588477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.589537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.589856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.590528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:10:52.590832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:52.592463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:52.605091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1180/2382] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:10:52.609552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce3b5a-52fb-4f64-9ab8-bbe9e0b3c6c8 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:52.619331 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce3b5a-52fb-4f64-9ab8-bbe9e0b3c6c8 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:52.624161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:52.624772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.625052 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.625322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.626021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:52.627975 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:52.718971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1203/2383] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:10:52 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:52.723232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:52.733295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ab798-8348-424d-8443-a2b0d1e532b3 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:52.788586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:52.789211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.789480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.789729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.790488 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:52.792455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:52.878180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1181/2384] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:10:52 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:10:52.882261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:52.893493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb8fa8-664a-46ae-9154-c689229a304d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:52.898132 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:52.898729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.900857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.900857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.900857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:52.901497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:52.911602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1204/2385] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:52.915428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:52.925218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38513448-ba91-4662-9ca8-a5ed2622de04 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:52.929822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454413f-6a3f-43e4-8a23-354f96feec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:52.930414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454413f-6a3f-43e4-8a23-354f96feec7c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.930695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454413f-6a3f-43e4-8a23-354f96feec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.930968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454413f-6a3f-43e4-8a23-354f96feec7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.931612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454413f-6a3f-43e4-8a23-354f96feec7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:52.933681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454413f-6a3f-43e4-8a23-354f96feec7c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:52.944052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1182/2386] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:10:52.948595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:52.958621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63a0da-9d68-4c75-bfd9-1213c0eb7453 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:52.963233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:52.964024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:52.964401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:52.964682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] PATH_INFO: `/v3/projects/9ff9a5319630429488810e66440111b9/users/5d34a0d281eb4b0b9da5b528d500f365/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:52.978830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff9a5319630429488810e66440111b9, user_id=5d34a0d281eb4b0b9da5b528d500f365, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:52.981424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:53.042937 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1205/2387] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:10:52 2022] PUT /identity/v3/projects/9ff9a5319630429488810e66440111b9/users/5d34a0d281eb4b0b9da5b528d500f365/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:10:53.054338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20d1c0-9485-49a8-9c4c-be7a24133031 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:53.054624 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20d1c0-9485-49a8-9c4c-be7a24133031 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:53.054923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20d1c0-9485-49a8-9c4c-be7a24133031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:53.083308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-df20d1c0-9485-49a8-9c4c-be7a24133031 None None] MFA Rules not processed for user `5d34a0d281eb4b0b9da5b528d500f365`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:53.084506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-df20d1c0-9485-49a8-9c4c-be7a24133031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:53.146666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1183/2388] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:53.157932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:53.159704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:53.219846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3416716-2bc2-48af-ac16-9a9f2a03cd36 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:53.223653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:53.224270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:53.224549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:53.224920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:53.237375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:53.238760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:53.248954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1206/2389] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:10:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:10:54.167487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75dc61-c30a-4303-99c5-0db3fc867a3c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:54.168121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75dc61-c30a-4303-99c5-0db3fc867a3c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:54.168121 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75dc61-c30a-4303-99c5-0db3fc867a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:54.181529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:10:54.189576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6b75dc61-c30a-4303-99c5-0db3fc867a3c None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:10:54.191556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6b75dc61-c30a-4303-99c5-0db3fc867a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:54.249328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1184/2390] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:10:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:10:54.377273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:10:54.378776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:10:54.460766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961dabc-d8e8-4504-8c51-ab01f77bc155 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:10:54.466750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8177b-6e87-4815-9679-734bd4e3fa03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:10:54.467359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8177b-6e87-4815-9679-734bd4e3fa03 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:10:54.467635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8177b-6e87-4815-9679-734bd4e3fa03 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:10:54.467904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8177b-6e87-4815-9679-734bd4e3fa03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:10:54.476682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8177b-6e87-4815-9679-734bd4e3fa03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:10:54.478366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8177b-6e87-4815-9679-734bd4e3fa03 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:10:54.487431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1207/2391] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:10:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:11:04.304748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=22 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:04.484028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:08.680326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0d89b-fee1-4966-b278-8ecfe63b1ea4 None None] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:08.681203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0d89b-fee1-4966-b278-8ecfe63b1ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:08.681203 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0d89b-fee1-4966-b278-8ecfe63b1ea4 None None] PATH_INFO: `/` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:08.682797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1185/2392] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:11:08.690342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e5ac3-26c6-43d9-895c-ef996f76d19e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:08.690882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e5ac3-26c6-43d9-895c-ef996f76d19e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:08.691190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e5ac3-26c6-43d9-895c-ef996f76d19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:08.713645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-374e5ac3-26c6-43d9-895c-ef996f76d19e None None] MFA Rules not processed for user `4ed50a15319e4991ae90b0e5271f4c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:08.715573 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-374e5ac3-26c6-43d9-895c-ef996f76d19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:08.776543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1208/2393] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:11:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:08.790680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0d89b-fee1-4966-b278-8ecfe63b1ea4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:08.792975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cea0d89b-fee1-4966-b278-8ecfe63b1ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:08.806647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0d89b-fee1-4966-b278-8ecfe63b1ea4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:08.811564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c56ba-5fa0-413f-b819-893fe69f0084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:08.812165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c56ba-5fa0-413f-b819-893fe69f0084 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:08.812447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c56ba-5fa0-413f-b819-893fe69f0084 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:08.812720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c56ba-5fa0-413f-b819-893fe69f0084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:08.819886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c56ba-5fa0-413f-b819-893fe69f0084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:08.821244 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c56ba-5fa0-413f-b819-893fe69f0084 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:08.829523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1186/2394] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:08.911715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43de01-29ba-4720-be52-239514eef19f None None] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:08.912009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43de01-29ba-4720-be52-239514eef19f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:08.912286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43de01-29ba-4720-be52-239514eef19f None None] PATH_INFO: `/` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:08.913660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1209/2395] 10.222.0.72 () {54 vars in 921 bytes} [Tue Sep 13 13:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:11:08.919639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18fc3f-ef8c-4c1a-abbb-d69602a8262d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:08.919926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18fc3f-ef8c-4c1a-abbb-d69602a8262d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:08.920273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18fc3f-ef8c-4c1a-abbb-d69602a8262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:08.940659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1b18fc3f-ef8c-4c1a-abbb-d69602a8262d None None] MFA Rules not processed for user `3b21d991d02b4f95a8d0fcec0a334495`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:08.941917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1b18fc3f-ef8c-4c1a-abbb-d69602a8262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:09.008423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1187/2396] 10.222.0.72 () {58 vars in 1018 bytes} [Tue Sep 13 13:11:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:09.019513 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43de01-29ba-4720-be52-239514eef19f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:09.031155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43de01-29ba-4720-be52-239514eef19f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:09.034791 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:09.035714 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:09.035995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:09.036358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:09.047545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:09.048958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:09.060511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1210/2397] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:17.107287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6ae94-cf9a-4a53-ba9a-bd7194099430 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:17.107767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6ae94-cf9a-4a53-ba9a-bd7194099430 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:17.107861 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6ae94-cf9a-4a53-ba9a-bd7194099430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:17.127054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-25f6ae94-cf9a-4a53-ba9a-bd7194099430 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:17.128234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-25f6ae94-cf9a-4a53-ba9a-bd7194099430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:17.135199 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1188/2398] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:17.147752 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:17.160166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823058e-8b31-4046-990a-3b130269bced service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:17.165314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4d3e4-3510-49c8-bd33-77ea8dd577db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:17.165953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d3e4-3510-49c8-bd33-77ea8dd577db service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:17.166264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d3e4-3510-49c8-bd33-77ea8dd577db service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:17.166575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d3e4-3510-49c8-bd33-77ea8dd577db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:17.174517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4d3e4-3510-49c8-bd33-77ea8dd577db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:17.176174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4d3e4-3510-49c8-bd33-77ea8dd577db service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:17.184154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1211/2399] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:11:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 13:11:18.379872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6ae94-cf9a-4a53-ba9a-bd7194099430 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:18.457722 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6ae94-cf9a-4a53-ba9a-bd7194099430 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:18.463030 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:18.463706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:18.464143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:18.464480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] PATH_INFO: `/v3/users/5d34a0d281eb4b0b9da5b528d500f365` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:18.465778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d34a0d281eb4b0b9da5b528d500f365)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:18.467898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:18.668882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d34a0d281eb4b0b9da5b528d500f365'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:18.669615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d34a0d281eb4b0b9da5b528d500f365'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:18.676828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d34a0d281eb4b0b9da5b528d500f365'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:18.694170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d34a0d281eb4b0b9da5b528d500f365'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:18.695176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d34a0d281eb4b0b9da5b528d500f365'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:18.697057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:18.754791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1189/2400] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:18 2022] DELETE /identity/v3/users/5d34a0d281eb4b0b9da5b528d500f365 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:18.759346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:18.759841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:18.764269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:18.780857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:18.782189 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:18.874453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1212/2401] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:18.884458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:18.886626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:18.948506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70a4f4-c9f5-41ec-a948-648b922b61f9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:18.952436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd5b60-15ea-48b3-8d91-52df9a05b0af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:18.954695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd5b60-15ea-48b3-8d91-52df9a05b0af service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:18.954695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd5b60-15ea-48b3-8d91-52df9a05b0af service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:18.954695 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd5b60-15ea-48b3-8d91-52df9a05b0af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:18.969161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd5b60-15ea-48b3-8d91-52df9a05b0af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:18.970647 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd5b60-15ea-48b3-8d91-52df9a05b0af service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:18.981791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1190/2402] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:19.432817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:19.442857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:19.455434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c825a-388c-4262-acf7-fded721aa09f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:19.461122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:19.462160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:19.462600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:19.463080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] PATH_INFO: `/v3/projects/9ff9a5319630429488810e66440111b9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:19.466180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff9a5319630429488810e66440111b9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:19.468999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:19.652177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff9a5319630429488810e66440111b9', 'request_id': 'req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:19.732920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff9a5319630429488810e66440111b9', 'request_id': 'req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:19.741297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff9a5319630429488810e66440111b9', 'request_id': 'req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:19.753492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] The token cache is being invalidate because project 9ff9a5319630429488810e66440111b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:11:19.753839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:19.754523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d01adfc8-b680-4a0e-acc9-41f0c3c4e2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:19.759305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1213/2403] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:11:19 2022] DELETE /identity/v3/projects/9ff9a5319630429488810e66440111b9 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:19.761477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73602340-2d6b-4fcf-8b24-4338fcbf0b18 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:19.765598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73602340-2d6b-4fcf-8b24-4338fcbf0b18 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:19.765598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73602340-2d6b-4fcf-8b24-4338fcbf0b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:19.790305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-73602340-2d6b-4fcf-8b24-4338fcbf0b18 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:19.791671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-73602340-2d6b-4fcf-8b24-4338fcbf0b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:19.855853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1191/2404] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:19.863581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539355b-cdf6-4099-9321-4405d9c1b3bf None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:19.864165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539355b-cdf6-4099-9321-4405d9c1b3bf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:19.864165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539355b-cdf6-4099-9321-4405d9c1b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:19.881044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:19.889312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7539355b-cdf6-4099-9321-4405d9c1b3bf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:19.890921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7539355b-cdf6-4099-9321-4405d9c1b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:19.904151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1214/2405] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:11:19.909277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73602340-2d6b-4fcf-8b24-4338fcbf0b18 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:19.922243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73602340-2d6b-4fcf-8b24-4338fcbf0b18 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:19.929874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:19.930636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:19.930985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:19.931270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:19.936432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:11:19.936804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:19.938521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:19.953865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1192/2406] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:11:19.956459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539355b-cdf6-4099-9321-4405d9c1b3bf None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:19.966302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539355b-cdf6-4099-9321-4405d9c1b3bf None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:19.971452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:19.972197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:19.972871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:19.973196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:19.973920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:19.975883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:20.078312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1215/2407] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:11:19 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:11:20.082628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:20.093723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c759b39-45d5-4dc5-9eb9-a625a370eed5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:20.154142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:20.155057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:20.155386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:20.155663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:20.156312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:20.158242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:20.249657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1193/2408] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:11:20 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:11:20.254036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:20.264922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d30e8-4c54-4bd6-9bf2-a69c9d5dfa3f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:20.269948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:20.276027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:20.276027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:20.276027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:20.276027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:20.276027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:20.285781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1216/2409] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:20.289887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:20.300908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d882dc-1fa6-4a48-bbfd-ed98f49c327e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:20.305626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429c28d-3158-4ac1-b1d1-efa8f7513607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:20.306394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429c28d-3158-4ac1-b1d1-efa8f7513607 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:20.306745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429c28d-3158-4ac1-b1d1-efa8f7513607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:20.307103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429c28d-3158-4ac1-b1d1-efa8f7513607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:20.308472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429c28d-3158-4ac1-b1d1-efa8f7513607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:20.314373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429c28d-3158-4ac1-b1d1-efa8f7513607 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:20.321783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:20.329204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1194/2410] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:20.333403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:20.345779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea48a2-c81e-42ea-80c3-dae66258f869 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:20.351328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:20.352095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:20.352457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:20.352803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] PATH_INFO: `/v3/projects/606ef4957160405eb02950a7b6e7c5dc/users/647db0116fba4fefa7d302a068c01ecb/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:20.371204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=606ef4957160405eb02950a7b6e7c5dc, user_id=647db0116fba4fefa7d302a068c01ecb, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:20.374060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:20.467097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1217/2411] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:11:20 2022] PUT /identity/v3/projects/606ef4957160405eb02950a7b6e7c5dc/users/647db0116fba4fefa7d302a068c01ecb/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:20.478979 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db140a-fe88-4736-a729-b73659123da8 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:20.479324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db140a-fe88-4736-a729-b73659123da8 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:20.480160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db140a-fe88-4736-a729-b73659123da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:20.509946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-81db140a-fe88-4736-a729-b73659123da8 None None] MFA Rules not processed for user `647db0116fba4fefa7d302a068c01ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:20.511259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-81db140a-fe88-4736-a729-b73659123da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:20.581875 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1195/2412] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:20.594285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:20.595984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:20.690577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04348266-9399-40a9-a1d7-ea94c0098d37 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:20.694167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81868be-2c6a-4dc7-b1b8-51e2c07b1589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:20.694781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81868be-2c6a-4dc7-b1b8-51e2c07b1589 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:20.695067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81868be-2c6a-4dc7-b1b8-51e2c07b1589 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:20.695345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81868be-2c6a-4dc7-b1b8-51e2c07b1589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:20.708327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81868be-2c6a-4dc7-b1b8-51e2c07b1589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:20.709851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81868be-2c6a-4dc7-b1b8-51e2c07b1589 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:20.720124 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1218/2413] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:24.929562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db140a-fe88-4736-a729-b73659123da8 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:24.994347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db140a-fe88-4736-a729-b73659123da8 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:24.999388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:25.000028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:25.000305 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:25.000580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] PATH_INFO: `/v3/users/647db0116fba4fefa7d302a068c01ecb` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:25.001723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=647db0116fba4fefa7d302a068c01ecb)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:25.003662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:25.239006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '647db0116fba4fefa7d302a068c01ecb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:25.239578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '647db0116fba4fefa7d302a068c01ecb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:25.248349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '647db0116fba4fefa7d302a068c01ecb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:25.270938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '647db0116fba4fefa7d302a068c01ecb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:25.272367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '647db0116fba4fefa7d302a068c01ecb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:25.345915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1196/2414] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:24 2022] DELETE /identity/v3/users/647db0116fba4fefa7d302a068c01ecb => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:25.352080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:25.352673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:25.352673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:25.373918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:25.375685 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:25.442556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1219/2415] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:25.449518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:25.451435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:25.462595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:25.513319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df9d61-a3ed-45e7-89ce-e41035969180 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:25.517215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf20a02-b9aa-4684-98bf-e226f094ead9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:25.517896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf20a02-b9aa-4684-98bf-e226f094ead9 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:25.518174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf20a02-b9aa-4684-98bf-e226f094ead9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:25.518442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf20a02-b9aa-4684-98bf-e226f094ead9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:25.529198 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf20a02-b9aa-4684-98bf-e226f094ead9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:25.530888 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf20a02-b9aa-4684-98bf-e226f094ead9 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:25.543642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1197/2416] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:26.146780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:26.147100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:26.160681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b985ab-d045-4b51-8a28-dfb2ce8faf7c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:26.165587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:26.166287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.166581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.166994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] PATH_INFO: `/v3/projects/606ef4957160405eb02950a7b6e7c5dc` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.168303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=606ef4957160405eb02950a7b6e7c5dc)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:26.170686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:26.268904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:26.274959 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606ef4957160405eb02950a7b6e7c5dc', 'request_id': 'req-499084c3-f96f-40d3-91f0-0ad7c34fb5de'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:26.372009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606ef4957160405eb02950a7b6e7c5dc', 'request_id': 'req-499084c3-f96f-40d3-91f0-0ad7c34fb5de'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:26.385175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606ef4957160405eb02950a7b6e7c5dc', 'request_id': 'req-499084c3-f96f-40d3-91f0-0ad7c34fb5de'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:26.395223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] The token cache is being invalidate because project 606ef4957160405eb02950a7b6e7c5dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:11:26.395515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:26.396174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-499084c3-f96f-40d3-91f0-0ad7c34fb5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:26.397672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1220/2417] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:11:26 2022] DELETE /identity/v3/projects/606ef4957160405eb02950a7b6e7c5dc => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:26.402878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb704-15f0-4c80-89f7-816a9d3fef1e None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.408120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb704-15f0-4c80-89f7-816a9d3fef1e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.408120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eb704-15f0-4c80-89f7-816a9d3fef1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.423303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-be6eb704-15f0-4c80-89f7-816a9d3fef1e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:26.424466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-be6eb704-15f0-4c80-89f7-816a9d3fef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:26.485806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1198/2418] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:11:26.493806 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c98c81-eccc-4c9f-bcae-8b5fec204f42 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.494237 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c98c81-eccc-4c9f-bcae-8b5fec204f42 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.495431 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c98c81-eccc-4c9f-bcae-8b5fec204f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.515578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-23c98c81-eccc-4c9f-bcae-8b5fec204f42 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:26.516931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-23c98c81-eccc-4c9f-bcae-8b5fec204f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:26.524885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1221/2419] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:26.529502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eb704-15f0-4c80-89f7-816a9d3fef1e None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:26.539154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eb704-15f0-4c80-89f7-816a9d3fef1e None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:26.543913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:26.544501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.544773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.545039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.545674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:11:26.545997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:26.547557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:26.559289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1199/2420] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:11:26.564838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c98c81-eccc-4c9f-bcae-8b5fec204f42 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:26.575769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c98c81-eccc-4c9f-bcae-8b5fec204f42 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:26.580534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:26.581169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.581452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.581727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.582388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:26.584436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:26.661499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1222/2421] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:11:26 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:11:26.667318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:26.677334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232f9d1-9413-4725-9fe2-adf6ed781d93 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:26.733526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:26.734130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.734406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.734677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.735302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:26.737196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:26.824801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1200/2422] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:11:26 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:11:26.828928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:26.839347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab9161-20a4-4bce-9dd5-a973f32b8ccc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:26.843935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79727283-d20b-4cca-b351-448908712141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:26.844580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79727283-d20b-4cca-b351-448908712141 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.844859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79727283-d20b-4cca-b351-448908712141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.845150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79727283-d20b-4cca-b351-448908712141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.845794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79727283-d20b-4cca-b351-448908712141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:26.847390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79727283-d20b-4cca-b351-448908712141 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:26.858377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1223/2423] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:26.862328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:26.873145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595437e3-a038-4465-9fce-7c4da75a617e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:26.877864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b0fc6-8dce-4342-9d2c-17a8cb5e5e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:26.878458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b0fc6-8dce-4342-9d2c-17a8cb5e5e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.878730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b0fc6-8dce-4342-9d2c-17a8cb5e5e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.878999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b0fc6-8dce-4342-9d2c-17a8cb5e5e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.879637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b0fc6-8dce-4342-9d2c-17a8cb5e5e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:26.881209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b0fc6-8dce-4342-9d2c-17a8cb5e5e22 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:26.891791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1201/2424] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:26.895762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79727283-d20b-4cca-b351-448908712141 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:26.905677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79727283-d20b-4cca-b351-448908712141 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:26.910411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:26.911044 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:26.911321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:26.911594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] PATH_INFO: `/v3/projects/d65e24097e1a4359b89f067df0da9080/users/79e9b5311a12498986c9ac510a2b1490/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:26.926164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65e24097e1a4359b89f067df0da9080, user_id=79e9b5311a12498986c9ac510a2b1490, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:26.928746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:27.003785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1224/2425] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:11:26 2022] PUT /identity/v3/projects/d65e24097e1a4359b89f067df0da9080/users/79e9b5311a12498986c9ac510a2b1490/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:27.014316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b82949-72a2-4d2b-a3c0-1b32a7468080 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.015550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b82949-72a2-4d2b-a3c0-1b32a7468080 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.015550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b82949-72a2-4d2b-a3c0-1b32a7468080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.043609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-99b82949-72a2-4d2b-a3c0-1b32a7468080 None None] MFA Rules not processed for user `79e9b5311a12498986c9ac510a2b1490`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:27.044970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-99b82949-72a2-4d2b-a3c0-1b32a7468080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:27.050120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:27.119448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1202/2426] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:27.124321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:27.193760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee727c4-cdba-4604-97d0-c16cf9e52ac5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:27.199295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:27.200155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.200408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.200730 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.201407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:27.203323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:27.301152 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1225/2427] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:11:27 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:11:27.305251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b82949-72a2-4d2b-a3c0-1b32a7468080 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:27.316348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b82949-72a2-4d2b-a3c0-1b32a7468080 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:27.372065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:27.372289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.372568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.372841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.373466 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:27.375420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:27.488255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1203/2428] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:11:27 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:11:27.492255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:27.502786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de24a82-a92a-4a0c-9720-30cded00920d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:27.507416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:27.508009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.508298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.508572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.509214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:27.511482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:27.522902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1226/2429] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:27.527878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:27.538242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86f88-d2e6-4dd3-bba3-b1d006106677 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:27.543192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec4460-5d1f-4c92-b95d-0c16b6709efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:27.543857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec4460-5d1f-4c92-b95d-0c16b6709efa admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.544145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec4460-5d1f-4c92-b95d-0c16b6709efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.544422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec4460-5d1f-4c92-b95d-0c16b6709efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.545067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec4460-5d1f-4c92-b95d-0c16b6709efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:27.546693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec4460-5d1f-4c92-b95d-0c16b6709efa admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:27.556460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1204/2430] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:27.560622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:27.570470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb180510-76bf-47b3-ad1b-ade4956caf17 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:27.575742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:27.576387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.576667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.576942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] PATH_INFO: `/v3/projects/f2a57c2eae2f4d49b02f43ee5aaabba2/users/e15300fff106459394674e5eb540af2c/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.591994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a57c2eae2f4d49b02f43ee5aaabba2, user_id=e15300fff106459394674e5eb540af2c, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:27.594675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:27.717256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1227/2431] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:11:27 2022] PUT /identity/v3/projects/f2a57c2eae2f4d49b02f43ee5aaabba2/users/e15300fff106459394674e5eb540af2c/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:27.728289 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade1c7-c6e1-4b63-a555-0e3827c88f3f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.733073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade1c7-c6e1-4b63-a555-0e3827c88f3f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.733073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ade1c7-c6e1-4b63-a555-0e3827c88f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.756446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-85ade1c7-c6e1-4b63-a555-0e3827c88f3f None None] MFA Rules not processed for user `e15300fff106459394674e5eb540af2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:27.757660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-85ade1c7-c6e1-4b63-a555-0e3827c88f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:27.820467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1205/2432] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:27.830678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:27.848769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:27.892916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558b7fe-18ea-432b-b8f3-09c5bd4c7833 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:27.897523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:27.901245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:27.901245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:27.901245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:27.901245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:27.901245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:27.996732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1228/2433] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:11:27 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:11:28.001186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ade1c7-c6e1-4b63-a555-0e3827c88f3f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:28.013124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ade1c7-c6e1-4b63-a555-0e3827c88f3f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:28.025138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:28.069709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:28.070337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:28.070617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:28.070893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:28.071517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:28.074172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:28.200945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1206/2434] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:11:28 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:11:28.205058 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:28.215377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256657-7308-4504-b30b-0d5a461b882e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:28.220047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:28.220641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:28.220919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:28.221193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:28.221869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:28.223489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:28.234966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1229/2435] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:28.244102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:28.255346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604406-f3c8-4e75-b9c7-4e3665247d9e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:28.259953 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:28.260686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:28.261220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:28.261220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] PATH_INFO: `/v3/projects/0ee49b67798d48418a8f6d2bf484f49e/users/980f5d3509a740fb800928584f49a5b6/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:28.276014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee49b67798d48418a8f6d2bf484f49e, user_id=980f5d3509a740fb800928584f49a5b6, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:28.278860 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:28.391309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1207/2436] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:11:28 2022] PUT /identity/v3/projects/0ee49b67798d48418a8f6d2bf484f49e/users/980f5d3509a740fb800928584f49a5b6/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:28.395212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:28.406434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d17dfb-d740-4b8c-af2f-a9ee6818b5a9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:28.481618 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5246f-7845-4db1-9ae8-ebc6409732aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:28.489900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5246f-7845-4db1-9ae8-ebc6409732aa admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:28.489900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5246f-7845-4db1-9ae8-ebc6409732aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:28.489900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5246f-7845-4db1-9ae8-ebc6409732aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:28.489900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5246f-7845-4db1-9ae8-ebc6409732aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:28.489900 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5246f-7845-4db1-9ae8-ebc6409732aa admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:28.505457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1230/2437] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:11:28.510176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:28.531264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af0946-e692-4162-99bf-2872a78afc6f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:28.534218 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:28.535184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:28.535489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:28.535767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] PATH_INFO: `/v3/projects/0ee49b67798d48418a8f6d2bf484f49e/users/980f5d3509a740fb800928584f49a5b6/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:28.537755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee49b67798d48418a8f6d2bf484f49e, user_id=980f5d3509a740fb800928584f49a5b6, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:28.541060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:28.554614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1208/2438] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:11:28 2022] PUT /identity/v3/projects/0ee49b67798d48418a8f6d2bf484f49e/users/980f5d3509a740fb800928584f49a5b6/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:11:28.566599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:28.566889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:28.567164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:28.596046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] MFA Rules not processed for user `980f5d3509a740fb800928584f49a5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:28.597261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:28.665761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1231/2439] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:28.677294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:28.678977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:28.739234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11732e1b-94e9-45d3-a6bd-bb9fb80c060c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:28.742761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:28.743353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:28.743629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:28.743916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:28.800109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:28.801543 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:28.810890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1209/2440] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:30.680107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:30.681520 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:30.693305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f42de8-293d-463b-816c-cef105a2f895 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:30.696913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a83be-af75-46e2-8984-700ef1c691a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:30.703246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a83be-af75-46e2-8984-700ef1c691a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:30.703246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a83be-af75-46e2-8984-700ef1c691a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:30.703246 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a83be-af75-46e2-8984-700ef1c691a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:30.707621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:30.757065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a83be-af75-46e2-8984-700ef1c691a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:30.758781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a83be-af75-46e2-8984-700ef1c691a0 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:30.769193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1232/2441] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:33.337663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:33.347999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31b9a2-847b-4a3e-9ca4-e6c8c798c910 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:33.351832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:33.352500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:33.352843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:33.353177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:33.362809 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:33.364410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:33.376088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1210/2442] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:35.852441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827e609-48d6-4bd8-bce1-4d5cd7683238 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:35.852441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827e609-48d6-4bd8-bce1-4d5cd7683238 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:35.852441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827e609-48d6-4bd8-bce1-4d5cd7683238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:35.875013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e827e609-48d6-4bd8-bce1-4d5cd7683238 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:35.876264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e827e609-48d6-4bd8-bce1-4d5cd7683238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:35.936753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1233/2443] 10.222.0.72 () {58 vars in 967 bytes} [Tue Sep 13 13:11:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:35.950482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:35.952153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:35.963309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:36.023265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a664a-1e23-4a76-a2dc-be97ee249b1e service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:36.028649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b7059-ffa5-4494-adb7-bc5c6a00d0ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:36.029263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b7059-ffa5-4494-adb7-bc5c6a00d0ef service swift] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:36.029545 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b7059-ffa5-4494-adb7-bc5c6a00d0ef service swift] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:36.029852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b7059-ffa5-4494-adb7-bc5c6a00d0ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:36.038974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b7059-ffa5-4494-adb7-bc5c6a00d0ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:36.040588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b7059-ffa5-4494-adb7-bc5c6a00d0ef service swift] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:36.049224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1211/2444] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:11:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:11:38.489359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827e609-48d6-4bd8-bce1-4d5cd7683238 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:38.566126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827e609-48d6-4bd8-bce1-4d5cd7683238 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:38.578775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:38.579445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:38.579796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:38.580156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] PATH_INFO: `/v3/users/79e9b5311a12498986c9ac510a2b1490` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:38.581425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e9b5311a12498986c9ac510a2b1490)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:38.583917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:38.825106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e9b5311a12498986c9ac510a2b1490'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:38.825890 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e9b5311a12498986c9ac510a2b1490'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:38.832831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e9b5311a12498986c9ac510a2b1490'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:38.834838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:38.852604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e9b5311a12498986c9ac510a2b1490'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:38.853336 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e9b5311a12498986c9ac510a2b1490'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:38.968252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1234/2445] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:38 2022] DELETE /identity/v3/users/79e9b5311a12498986c9ac510a2b1490 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:38.972820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:38.973303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:38.973552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:38.994079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:38.995304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:39.060824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1212/2446] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:39.070754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:39.072212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:39.131391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523a5d3-1d9a-47f5-bc22-832d6ee86085 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:39.135185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:39.135789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:39.136096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:39.136359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:39.146130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:39.147555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:39.160723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1235/2447] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:39.583472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:39.583472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:39.598619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc5982-ca89-4130-b654-7119482318e3 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:39.604949 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:39.605583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:39.605889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:39.606161 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] PATH_INFO: `/v3/projects/d65e24097e1a4359b89f067df0da9080` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:39.610130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65e24097e1a4359b89f067df0da9080)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:39.610130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:39.710795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65e24097e1a4359b89f067df0da9080', 'request_id': 'req-6a1396d7-341e-43c1-8acc-d55b66e5314f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:39.784852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65e24097e1a4359b89f067df0da9080', 'request_id': 'req-6a1396d7-341e-43c1-8acc-d55b66e5314f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:39.791804 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65e24097e1a4359b89f067df0da9080', 'request_id': 'req-6a1396d7-341e-43c1-8acc-d55b66e5314f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:39.802162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] The token cache is being invalidate because project d65e24097e1a4359b89f067df0da9080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:11:39.802448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:39.803077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:39.804537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1213/2448] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:11:39 2022] DELETE /identity/v3/projects/d65e24097e1a4359b89f067df0da9080 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:39.808590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:39.818797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992211e-b749-483c-bd31-3291d828705b service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:39.880846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:39.881425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:39.881711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:39.882015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] PATH_INFO: `/v3/users/e15300fff106459394674e5eb540af2c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:39.883148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15300fff106459394674e5eb540af2c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:39.885066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:40.042478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15300fff106459394674e5eb540af2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:40.043149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15300fff106459394674e5eb540af2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:40.050858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15300fff106459394674e5eb540af2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:40.063125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15300fff106459394674e5eb540af2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:40.063768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15300fff106459394674e5eb540af2c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:40.067360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:40.167578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1236/2449] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:39 2022] DELETE /identity/v3/users/e15300fff106459394674e5eb540af2c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:40.655798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:40.658662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:40.685513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1396d7-341e-43c1-8acc-d55b66e5314f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:40.717791 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:40.783186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:40.783835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:40.784414 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:40.784648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] PATH_INFO: `/v3/projects/f2a57c2eae2f4d49b02f43ee5aaabba2` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:40.785835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a57c2eae2f4d49b02f43ee5aaabba2)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:40.787824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:40.898862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a57c2eae2f4d49b02f43ee5aaabba2', 'request_id': 'req-80a6008c-39c3-4e8e-9790-2a63bb490b1d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.012468 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a57c2eae2f4d49b02f43ee5aaabba2', 'request_id': 'req-80a6008c-39c3-4e8e-9790-2a63bb490b1d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.019755 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a57c2eae2f4d49b02f43ee5aaabba2', 'request_id': 'req-80a6008c-39c3-4e8e-9790-2a63bb490b1d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.030276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] The token cache is being invalidate because project f2a57c2eae2f4d49b02f43ee5aaabba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:11:41.030564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.031781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.033257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1214/2450] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:11:40 2022] DELETE /identity/v3/projects/f2a57c2eae2f4d49b02f43ee5aaabba2 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:41.037659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:41.039127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:41.052323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b24159-f48e-490b-8ad2-d78d1ccc7b01 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:41.107197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:41.107817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:41.108113 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:41.108385 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] PATH_INFO: `/v3/users/980f5d3509a740fb800928584f49a5b6` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:41.110080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=980f5d3509a740fb800928584f49a5b6)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:41.111980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:41.398999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980f5d3509a740fb800928584f49a5b6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.399746 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980f5d3509a740fb800928584f49a5b6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.410167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980f5d3509a740fb800928584f49a5b6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.423177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980f5d3509a740fb800928584f49a5b6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.423945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c17b0300-a014-40a2-aade-bab6af21f3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980f5d3509a740fb800928584f49a5b6'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:41.493244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1237/2451] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:41 2022] DELETE /identity/v3/users/980f5d3509a740fb800928584f49a5b6 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:41.949359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:41.957903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:41.964915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6008c-39c3-4e8e-9790-2a63bb490b1d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:42.022355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:42.022976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.023249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.023518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] PATH_INFO: `/v3/projects/0ee49b67798d48418a8f6d2bf484f49e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.024614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee49b67798d48418a8f6d2bf484f49e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:42.026547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:42.137344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee49b67798d48418a8f6d2bf484f49e', 'request_id': 'req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:42.141386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:42.239243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee49b67798d48418a8f6d2bf484f49e', 'request_id': 'req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:42.246144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee49b67798d48418a8f6d2bf484f49e', 'request_id': 'req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:42.261562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] The token cache is being invalidate because project 0ee49b67798d48418a8f6d2bf484f49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:11:42.262187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:42.263292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e78c75a2-826e-4f73-b74e-6d5c3f4c32b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:42.264798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1215/2452] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:11:41 2022] DELETE /identity/v3/projects/0ee49b67798d48418a8f6d2bf484f49e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:11:42.269699 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3e8e9-6e4f-4886-83dd-edbf322435e9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.270180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3e8e9-6e4f-4886-83dd-edbf322435e9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.270273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3e8e9-6e4f-4886-83dd-edbf322435e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.290853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-8bb3e8e9-6e4f-4886-83dd-edbf322435e9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:42.292049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8bb3e8e9-6e4f-4886-83dd-edbf322435e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:42.350892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1238/2453] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:42.415916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab233785-f0f3-40fc-bd71-68a8209607f5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.416109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab233785-f0f3-40fc-bd71-68a8209607f5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.416382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab233785-f0f3-40fc-bd71-68a8209607f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.436486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ab233785-f0f3-40fc-bd71-68a8209607f5 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:42.437693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ab233785-f0f3-40fc-bd71-68a8209607f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:42.445514 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1216/2454] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:42.449664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3e8e9-6e4f-4886-83dd-edbf322435e9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:42.460415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3e8e9-6e4f-4886-83dd-edbf322435e9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:42.465075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:42.465695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.466015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.466285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.466923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:11:42.467234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:42.469625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:42.481282 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1239/2455] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:11:42.486478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab233785-f0f3-40fc-bd71-68a8209607f5 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:42.495800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab233785-f0f3-40fc-bd71-68a8209607f5 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:42.500532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:42.501130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.501411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.501683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.502375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:42.504360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:42.592756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1217/2456] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:11:42 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:11:42.598362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:42.604740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:42.617398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451768b-0494-4603-8a2e-3dbe41bd1b7d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:42.681108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:42.681638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.681934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.682214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.682855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:42.684795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:42.773595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1240/2457] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:11:42 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:11:42.778003 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:42.788880 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab2427-d4de-40bd-a59a-1e218fb73074 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:42.793420 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:42.794055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.794331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.794599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.795232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:42.797445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:42.807073 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1218/2458] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:42.810966 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:42.821075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006eab4-454c-450a-8f8f-243606cd1fe0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:42.825826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcbc79-d36b-42d4-ad38-96970a36a4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:42.826415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcbc79-d36b-42d4-ad38-96970a36a4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.826693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcbc79-d36b-42d4-ad38-96970a36a4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.826961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcbc79-d36b-42d4-ad38-96970a36a4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.827596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcbc79-d36b-42d4-ad38-96970a36a4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:42.829766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcbc79-d36b-42d4-ad38-96970a36a4cd admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:42.840572 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1241/2459] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:42.845159 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:42.854841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf2972-c124-4929-a7d1-18e82aa3313c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:42.859519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:42.860194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.860472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.860743 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] PATH_INFO: `/v3/projects/8b2eb2e872fe4451b020565cd7118244/users/b00d3c18727d49aa8b38ea47b01f70c3/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.874944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2eb2e872fe4451b020565cd7118244, user_id=b00d3c18727d49aa8b38ea47b01f70c3, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:42.877612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:42.950188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1219/2460] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:11:42 2022] PUT /identity/v3/projects/8b2eb2e872fe4451b020565cd7118244/users/b00d3c18727d49aa8b38ea47b01f70c3/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:11:42.960931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cef832-af16-4825-80ff-5417fb8797b3 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:42.961245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cef832-af16-4825-80ff-5417fb8797b3 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:42.961525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cef832-af16-4825-80ff-5417fb8797b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:42.989556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-28cef832-af16-4825-80ff-5417fb8797b3 None None] MFA Rules not processed for user `b00d3c18727d49aa8b38ea47b01f70c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:42.990796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-28cef832-af16-4825-80ff-5417fb8797b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:43.053948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1242/2461] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:43.064792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:43.066516 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:43.123251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598ddf9-946e-46f7-b752-002e164bec25 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:43.126897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0679702-c848-4ad7-aeea-044af93371a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:43.127487 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0679702-c848-4ad7-aeea-044af93371a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:43.127763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0679702-c848-4ad7-aeea-044af93371a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:43.128035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0679702-c848-4ad7-aeea-044af93371a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:43.136056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0679702-c848-4ad7-aeea-044af93371a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:43.137465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0679702-c848-4ad7-aeea-044af93371a0 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:43.146582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1220/2462] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:43.171507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cef832-af16-4825-80ff-5417fb8797b3 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:43.235869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cef832-af16-4825-80ff-5417fb8797b3 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:43.237326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:43.237994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:43.238275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:43.238551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] PATH_INFO: `/v3/users/b00d3c18727d49aa8b38ea47b01f70c3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:43.239686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00d3c18727d49aa8b38ea47b01f70c3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:43.241612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:43.256318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:43.432915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00d3c18727d49aa8b38ea47b01f70c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:43.433590 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00d3c18727d49aa8b38ea47b01f70c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:43.441343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00d3c18727d49aa8b38ea47b01f70c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:43.454370 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00d3c18727d49aa8b38ea47b01f70c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:43.455039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00d3c18727d49aa8b38ea47b01f70c3'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:43.526915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1243/2463] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:43 2022] DELETE /identity/v3/users/b00d3c18727d49aa8b38ea47b01f70c3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:43.531397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:43.531676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:43.531950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:43.551257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:43.552441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:43.611029 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1221/2464] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:11:43.621498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:43.622944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:43.685306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b2db8-50e5-418b-87b5-c981d8fb1559 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:43.689789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d236a1a-0d14-4fa9-b67b-958cb1334c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:43.690405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236a1a-0d14-4fa9-b67b-958cb1334c40 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:43.690681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236a1a-0d14-4fa9-b67b-958cb1334c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:43.690948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236a1a-0d14-4fa9-b67b-958cb1334c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:43.700155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d236a1a-0d14-4fa9-b67b-958cb1334c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:43.701581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d236a1a-0d14-4fa9-b67b-958cb1334c40 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:43.711098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1244/2465] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:44.272260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:44.273846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:44.286194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aaaa5d-feb4-46a1-8a21-7901d31cec49 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:44.291575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:44.303999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:44.303999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:44.303999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] PATH_INFO: `/v3/projects/8b2eb2e872fe4451b020565cd7118244` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:44.303999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2eb2e872fe4451b020565cd7118244)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:44.303999 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:44.315028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:44.445153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2eb2e872fe4451b020565cd7118244', 'request_id': 'req-64868fd6-a3f8-4b2b-b812-cd330d7e2717'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:44.517021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2eb2e872fe4451b020565cd7118244', 'request_id': 'req-64868fd6-a3f8-4b2b-b812-cd330d7e2717'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:44.524255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2eb2e872fe4451b020565cd7118244', 'request_id': 'req-64868fd6-a3f8-4b2b-b812-cd330d7e2717'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:44.536194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] The token cache is being invalidate because project 8b2eb2e872fe4451b020565cd7118244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:11:44.538653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:44.538653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-64868fd6-a3f8-4b2b-b812-cd330d7e2717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:44.538851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1222/2466] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:11:44 2022] DELETE /identity/v3/projects/8b2eb2e872fe4451b020565cd7118244 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:44.543579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332c3b-9ba5-465f-aed7-b1ee19b4a22c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:44.544099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332c3b-9ba5-465f-aed7-b1ee19b4a22c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:44.544099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f332c3b-9ba5-465f-aed7-b1ee19b4a22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:44.564776 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9f332c3b-9ba5-465f-aed7-b1ee19b4a22c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:44.566017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9f332c3b-9ba5-465f-aed7-b1ee19b4a22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:44.629342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1245/2467] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:44.638157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f27e74-f6e4-4858-80c9-968c15d6fb5d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:44.638402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f27e74-f6e4-4858-80c9-968c15d6fb5d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:44.638765 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f27e74-f6e4-4858-80c9-968c15d6fb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:44.661202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-68f27e74-f6e4-4858-80c9-968c15d6fb5d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:44.662585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-68f27e74-f6e4-4858-80c9-968c15d6fb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:44.675684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1223/2468] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:44.677104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332c3b-9ba5-465f-aed7-b1ee19b4a22c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:44.687095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f332c3b-9ba5-465f-aed7-b1ee19b4a22c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:44.691852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:44.692462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:44.692744 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:44.693097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:44.693672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:11:44.694042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:44.695833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:44.708396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1246/2469] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:11:44.712806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f27e74-f6e4-4858-80c9-968c15d6fb5d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:44.722939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f27e74-f6e4-4858-80c9-968c15d6fb5d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:44.727739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:44.728369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:44.728645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:44.728918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:44.729558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:44.731490 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:44.891679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1224/2470] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:11:44 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:11:44.896205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:44.907619 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bfa2-f19b-468c-ad3d-b311ea7428ef admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:44.926198 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:44.968108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:44.969564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:44.969873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:44.970148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:44.970818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:44.973704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:45.059338 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1247/2471] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:11:44 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:11:45.064395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:45.074960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f4cc4-f449-49b7-b5ec-b83cd6488987 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:45.080263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:45.080872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:45.081146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:45.081446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:45.082117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:45.083704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:45.094267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1225/2472] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:11:45.098568 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:45.108961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e52c02-50c2-44b2-bb21-03fd5f501dad admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:45.113799 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0615d-b538-4371-9cdd-fe2dc1b59a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:45.114406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0615d-b538-4371-9cdd-fe2dc1b59a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:45.114692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0615d-b538-4371-9cdd-fe2dc1b59a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:45.115078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0615d-b538-4371-9cdd-fe2dc1b59a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:45.115727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0615d-b538-4371-9cdd-fe2dc1b59a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:45.117419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0615d-b538-4371-9cdd-fe2dc1b59a03 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:45.132810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1248/2473] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:11:45.134292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:45.144811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e6242-7a94-4ebc-9b2d-c8de58eb7f2a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:45.149479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:45.150156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:45.150439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:45.150710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] PATH_INFO: `/v3/projects/15bcfc71833149c194401e5afb6e5d18/users/81634611141f4909a52ac46d5164f17f/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:45.165506 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bcfc71833149c194401e5afb6e5d18, user_id=81634611141f4909a52ac46d5164f17f, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:45.168189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:45.253842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1226/2474] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:11:45 2022] PUT /identity/v3/projects/15bcfc71833149c194401e5afb6e5d18/users/81634611141f4909a52ac46d5164f17f/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:45.259193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e96f7-a9f2-42b3-90df-eab01ec3343b None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:45.259482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e96f7-a9f2-42b3-90df-eab01ec3343b None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:45.267151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e96f7-a9f2-42b3-90df-eab01ec3343b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:45.280186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-988e96f7-a9f2-42b3-90df-eab01ec3343b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:45.281705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-988e96f7-a9f2-42b3-90df-eab01ec3343b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:45.349629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1249/2475] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:45.360027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:45.362082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:45.435599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bdfd3-0797-4613-a9d8-7056f2bb6920 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:45.440312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:45.441187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:45.441187 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:45.441767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:45.451236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:45.452714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:45.464427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1227/2476] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:47.020983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8d4d-aee0-4618-8e53-4983d633c261 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:47.021505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8d4d-aee0-4618-8e53-4983d633c261 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:47.021505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8d4d-aee0-4618-8e53-4983d633c261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:47.054104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-303a8d4d-aee0-4618-8e53-4983d633c261 None None] MFA Rules not processed for user `81634611141f4909a52ac46d5164f17f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:47.055902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-303a8d4d-aee0-4618-8e53-4983d633c261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:47.135587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1250/2477] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:11:47.146855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:47.148330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:47.210862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:47.223245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ae819-8309-4e17-9f54-af093325fd42 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:47.228903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:47.232864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:47.232864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:47.232864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:47.240581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:47.242488 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:47.253211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1228/2478] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:49.132222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35dd50-6356-479e-b70e-ae9907948374 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:49.132801 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35dd50-6356-479e-b70e-ae9907948374 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:49.132933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35dd50-6356-479e-b70e-ae9907948374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:49.155289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ef35dd50-6356-479e-b70e-ae9907948374 None None] MFA Rules not processed for user `81634611141f4909a52ac46d5164f17f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:49.156649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ef35dd50-6356-479e-b70e-ae9907948374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:49.164334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1251/2479] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:11:54.189893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:54.191495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:54.204295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6bcb8-d3e8-4a0e-b3a6-f0e4befd6748 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:54.208133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:54.208899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:54.209276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:54.209591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:54.211480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:54.224438 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:54.226063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:54.236625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1229/2480] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:54.500491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35dd50-6356-479e-b70e-ae9907948374 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:54.511718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35dd50-6356-479e-b70e-ae9907948374 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:54.515842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:54.516456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:54.517164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:54.517445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:54.520219 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:54.534229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:54.535648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:54.546941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1252/2481] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:58.726041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:58.730657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:58.802605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302283-26a4-4800-aa9f-5aa2d03b9691 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:58.807718 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:58.808331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:58.808612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:58.808887 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:58.810767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:58.826478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:58.829367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:58.842058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1230/2482] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:11:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:11:58.878587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:58.880193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:58.903176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31e277-9044-45da-b968-e523728669b1 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:58.914089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:58.914089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:58.914089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:58.914089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] PATH_INFO: `/v3/users/9fadb80f21614816b3526d59b2d8a0f7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:58.914554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fadb80f21614816b3526d59b2d8a0f7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:58.915194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:58.919463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:59.127716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fadb80f21614816b3526d59b2d8a0f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.129045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fadb80f21614816b3526d59b2d8a0f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.136992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fadb80f21614816b3526d59b2d8a0f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.154262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fadb80f21614816b3526d59b2d8a0f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.155426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fadb80f21614816b3526d59b2d8a0f7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.233533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1253/2483] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:58 2022] DELETE /identity/v3/users/9fadb80f21614816b3526d59b2d8a0f7 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:59.531851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:59.543633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc8e4a-a6dc-4e28-817f-534ac3c7f73b service placement] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:59.610255 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:59.614835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:59.614835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:59.614835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] PATH_INFO: `/v3/projects/7775bd9f19b64c8ca07407c103dcff9b` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:59.614835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7775bd9f19b64c8ca07407c103dcff9b)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:59.614835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:59.651949 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:11:59.653656 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:59.672489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecf471-8e55-4dc2-a165-8ca2fab7da2f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:11:59.676737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:11:59.677419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:59.677709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:59.678045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] PATH_INFO: `/v3/users/7b1c15ff53e5442881b56bf504b9a31b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:59.679185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1c15ff53e5442881b56bf504b9a31b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:11:59.681083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:11:59.738201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7775bd9f19b64c8ca07407c103dcff9b', 'request_id': 'req-eee6e86b-bb78-4aa5-9742-bc09097f88c9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.833138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7775bd9f19b64c8ca07407c103dcff9b', 'request_id': 'req-eee6e86b-bb78-4aa5-9742-bc09097f88c9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.840007 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7775bd9f19b64c8ca07407c103dcff9b', 'request_id': 'req-eee6e86b-bb78-4aa5-9742-bc09097f88c9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.844681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:11:59.851716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] The token cache is being invalidate because project 7775bd9f19b64c8ca07407c103dcff9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:11:59.852009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.852661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-eee6e86b-bb78-4aa5-9742-bc09097f88c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.854660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1231/2484] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:11:59 2022] DELETE /identity/v3/projects/7775bd9f19b64c8ca07407c103dcff9b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:11:59.860032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bf4a8-6b94-4f4b-9105-08604642ade3 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:59.860326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bf4a8-6b94-4f4b-9105-08604642ade3 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:59.860590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bf4a8-6b94-4f4b-9105-08604642ade3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:59.881532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-104bf4a8-6b94-4f4b-9105-08604642ade3 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:59.882754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-104bf4a8-6b94-4f4b-9105-08604642ade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:59.898679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1c15ff53e5442881b56bf504b9a31b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.899346 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1c15ff53e5442881b56bf504b9a31b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.907830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1c15ff53e5442881b56bf504b9a31b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.922159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1c15ff53e5442881b56bf504b9a31b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.924154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1c15ff53e5442881b56bf504b9a31b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:11:59.949980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1232/2485] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:11:59.958627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c4f13-7870-4eaf-b8d8-4366211136e7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:11:59.958911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c4f13-7870-4eaf-b8d8-4366211136e7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:11:59.959184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c4f13-7870-4eaf-b8d8-4366211136e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:11:59.979239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-a30c4f13-7870-4eaf-b8d8-4366211136e7 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:11:59.980465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a30c4f13-7870-4eaf-b8d8-4366211136e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:11:59.988616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1233/2486] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:11:59.992837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c4f13-7870-4eaf-b8d8-4366211136e7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.002439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c4f13-7870-4eaf-b8d8-4366211136e7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.007174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.007777 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.008058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.008333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.008984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:00.009288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.010828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1254/2487] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:11:59 2022] DELETE /identity/v3/users/7b1c15ff53e5442881b56bf504b9a31b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:00.016124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.027479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1234/2488] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:12:00.028780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.083357 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:00.093640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707aede-2c6a-4040-8583-e120d0fdad11 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.098546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.099647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.099931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.100211 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.100853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.102923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.223318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1255/2489] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:00.229214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.239522 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac1578-6f10-42d2-a83b-6fab93323bc7 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.292690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.294588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:00.322764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.328763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.328763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.328763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.328763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.331440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.377181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8e37d-4866-462e-a9eb-42573eea4333 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.382460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.383099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.383377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.383648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] PATH_INFO: `/v3/projects/75f6274158934ab5abf68951902f1e81` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.384768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f6274158934ab5abf68951902f1e81)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.386693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.442602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1235/2490] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:00.446733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.457100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d3cb9-09aa-4d99-8681-0a3bcaa820d9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.461721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.462334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.462612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.462884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.463554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.465158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.477781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1236/2491] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:00.481662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.492673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9c850-a056-42d9-90b5-b16cbdcaa128 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.497866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.498447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.498730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.499006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.500780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.502517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.512504 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1237/2492] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:00.517055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.526842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730bdf7-a941-4fd0-9a51-ff07bb780adc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.532028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779033a-2315-4976-81f9-76acd5f315bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.532650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779033a-2315-4976-81f9-76acd5f315bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.532944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779033a-2315-4976-81f9-76acd5f315bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.533196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779033a-2315-4976-81f9-76acd5f315bb admin admin] PATH_INFO: `/v3/projects/d374f2c2ecbb4aaa84709f9ccff19bc8/users/c32e36349ad947019eb2f531c61d981c/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.547997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779033a-2315-4976-81f9-76acd5f315bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d374f2c2ecbb4aaa84709f9ccff19bc8, user_id=c32e36349ad947019eb2f531c61d981c, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.550660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779033a-2315-4976-81f9-76acd5f315bb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.556977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f6274158934ab5abf68951902f1e81', 'request_id': 'req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:00.623670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1238/2493] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:00 2022] PUT /identity/v3/projects/d374f2c2ecbb4aaa84709f9ccff19bc8/users/c32e36349ad947019eb2f531c61d981c/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:00.634166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.634455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.634730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.642123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f6274158934ab5abf68951902f1e81', 'request_id': 'req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:00.649325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f6274158934ab5abf68951902f1e81', 'request_id': 'req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:00.661300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] The token cache is being invalidate because project 75f6274158934ab5abf68951902f1e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:00.662035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:00.662743 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-eb3fcc9e-d2d9-4570-a4ce-cba99fbe9138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:00.664502 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1256/2494] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:00 2022] DELETE /identity/v3/projects/75f6274158934ab5abf68951902f1e81 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:00.664781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] MFA Rules not processed for user `c32e36349ad947019eb2f531c61d981c`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:00.666085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:00.669196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ba248-6018-4f2d-8e8a-d486dafae503 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.669507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ba248-6018-4f2d-8e8a-d486dafae503 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.669885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ba248-6018-4f2d-8e8a-d486dafae503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.689150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-f94ba248-6018-4f2d-8e8a-d486dafae503 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:00.691859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f94ba248-6018-4f2d-8e8a-d486dafae503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:00.733694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1239/2495] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:00.738222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.739551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:00.751305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1257/2496] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:00.760032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb43302-326e-40a1-a012-bc3a13de3094 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.760318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb43302-326e-40a1-a012-bc3a13de3094 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.760591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb43302-326e-40a1-a012-bc3a13de3094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.781128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ceb43302-326e-40a1-a012-bc3a13de3094 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:00.782442 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ceb43302-326e-40a1-a012-bc3a13de3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:00.790613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1258/2497] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:00.794879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb43302-326e-40a1-a012-bc3a13de3094 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.804235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb43302-326e-40a1-a012-bc3a13de3094 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.809932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.810519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.810717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a95b67-b9f5-4885-aade-d97b69790b8c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.810838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.811056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.812232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:00.812709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.814309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.816076 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.816679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.818085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.818085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.818085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.820526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.826436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1259/2498] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:12:00.831388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.841477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09d29b-6966-4f2f-84c3-510bd7b2614b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:00.846768 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:00.847400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:00.847679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:00.847950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:00.848576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:00.850477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:00.905014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1240/2499] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:00.909443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.931729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:00.935673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1260/2500] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:00.940256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:00.975882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f01a02-46d3-4078-9242-513dad29cbf6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.005414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418fd4d-a098-4066-aa64-e7fe5c0cec31 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.011488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.013562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.013562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.013562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.013562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.016214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.032990 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:01.034439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.035052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.035329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.035597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.036323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.038257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.130864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1261/2501] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:01.134795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.144925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3d8a5-80c2-4aab-b89d-2b765bfb05cc admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.149539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.150262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.150440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.150713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.151406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.152964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.154623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1241/2502] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:00 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:01.160295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.166979 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1262/2503] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:01.168300 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.170671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62986d6-cfbc-4507-9943-094673dc13d6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.176233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.176886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.177103 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.177374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.178045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.178908 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c135a-457f-4109-bdd9-1ddf60d34255 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.179650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.183567 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.184196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.184479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.184755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.185398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.187663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.191935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1242/2504] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:01.198497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1263/2505] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:01.199967 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.202417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.210491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9dddd-5d98-4a5f-bc2e-14b115a2ee69 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.212351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dfd68-e7f6-4130-877d-42d24e0e37af admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.215512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.216392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.216684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.217368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce23719-d177-4525-8c42-1391572b8792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.217517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] PATH_INFO: `/v3/projects/868cf8d8e417423aa7143dd8c47551a5/users/01c2a96fedb74e7e82d97ec40aabb42e/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.218096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce23719-d177-4525-8c42-1391572b8792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.218381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce23719-d177-4525-8c42-1391572b8792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.218659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce23719-d177-4525-8c42-1391572b8792 admin admin] PATH_INFO: `/v3/projects/2d4c1a18f2c049169915a8f3fd8a57e7/users/461a89d41f844c19969adf1193c08327/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.233306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce23719-d177-4525-8c42-1391572b8792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4c1a18f2c049169915a8f3fd8a57e7, user_id=461a89d41f844c19969adf1193c08327, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.234800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] RBAC: Authorizing `identity:create_grant(project_id=868cf8d8e417423aa7143dd8c47551a5, user_id=01c2a96fedb74e7e82d97ec40aabb42e, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.236631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce23719-d177-4525-8c42-1391572b8792 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.237381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.304155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1264/2506] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:01 2022] PUT /identity/v3/projects/2d4c1a18f2c049169915a8f3fd8a57e7/users/461a89d41f844c19969adf1193c08327/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:01.317279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.317570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.317884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.321872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1243/2507] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:01 2022] PUT /identity/v3/projects/868cf8d8e417423aa7143dd8c47551a5/users/01c2a96fedb74e7e82d97ec40aabb42e/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:01.325672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.348810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] MFA Rules not processed for user `461a89d41f844c19969adf1193c08327`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:01.350076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:01.388234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc60cd-3391-4e24-b1f2-f8959aec161c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.393204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.393869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.394123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.394391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.395056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.396656 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.407101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1244/2508] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:01.411367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.418496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1265/2509] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:01.421425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1975a-1db9-4d66-be99-6868ee6e392b admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.426382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cb2ee-72ee-462f-b1ca-8b5c117bbac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.427040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cb2ee-72ee-462f-b1ca-8b5c117bbac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.427319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cb2ee-72ee-462f-b1ca-8b5c117bbac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.428046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cb2ee-72ee-462f-b1ca-8b5c117bbac0 admin admin] PATH_INFO: `/v3/projects/868cf8d8e417423aa7143dd8c47551a5/users/01c2a96fedb74e7e82d97ec40aabb42e/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.429479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cb2ee-72ee-462f-b1ca-8b5c117bbac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=868cf8d8e417423aa7143dd8c47551a5, user_id=01c2a96fedb74e7e82d97ec40aabb42e, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.432399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cb2ee-72ee-462f-b1ca-8b5c117bbac0 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.434239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.436003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:01.444074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1245/2510] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:01 2022] PUT /identity/v3/projects/868cf8d8e417423aa7143dd8c47551a5/users/01c2a96fedb74e7e82d97ec40aabb42e/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:01.455290 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.455571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.455842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.487659 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] MFA Rules not processed for user `01c2a96fedb74e7e82d97ec40aabb42e`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:01.488965 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:01.520907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e88c5-cceb-4ab0-af4d-36209ba04561 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.571303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1246/2511] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:01.584321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.585821 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:01.590377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.591002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.591285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.591708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.653274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f1a2-7cfc-40c2-8218-e3de8104fb42 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.660140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.665940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.665940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.665940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:01.666406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.668688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.674143 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:01.675710 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:01.682140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1266/2512] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:01.689363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1247/2513] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:01.892754 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:01.937322 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:01.948598 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e3c5d-71f3-49a0-acfd-b4d04bd3b496 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:01.953855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3fcd5-03ce-4645-a392-7d79e1f53e88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:01.954508 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3fcd5-03ce-4645-a392-7d79e1f53e88 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:01.954818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3fcd5-03ce-4645-a392-7d79e1f53e88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:01.955090 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3fcd5-03ce-4645-a392-7d79e1f53e88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:02.023542 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3fcd5-03ce-4645-a392-7d79e1f53e88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:02.024993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3fcd5-03ce-4645-a392-7d79e1f53e88 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:02.035602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1267/2514] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:02.087461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:02.915509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:02.987638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:13.128968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:13.193672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e428fb1-125c-4320-ae77-db31789b9881 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:13.198390 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:13.199022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:13.199331 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:13.199574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] PATH_INFO: `/v3/users/461a89d41f844c19969adf1193c08327` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:13.200790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=461a89d41f844c19969adf1193c08327)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:13.202940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:13.267485 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=12 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:13.432292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461a89d41f844c19969adf1193c08327'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:13.432974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461a89d41f844c19969adf1193c08327'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:13.445680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461a89d41f844c19969adf1193c08327'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:13.458894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461a89d41f844c19969adf1193c08327'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:13.459621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461a89d41f844c19969adf1193c08327'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:13.561273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1248/2515] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:13 2022] DELETE /identity/v3/users/461a89d41f844c19969adf1193c08327 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:13.566661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:13.566994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:13.567359 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:13.588104 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:13.589423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:13.652159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1268/2516] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:13.663057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:13.664693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:13.698600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:13.731914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36362c-5186-4771-8b1b-67e7a9ad9bf5 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:13.736408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a2b4a-0b5b-496d-9eaa-2882a2b37095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:13.737082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2b4a-0b5b-496d-9eaa-2882a2b37095 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:13.737358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2b4a-0b5b-496d-9eaa-2882a2b37095 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:13.737630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a2b4a-0b5b-496d-9eaa-2882a2b37095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:13.747959 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a2b4a-0b5b-496d-9eaa-2882a2b37095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:13.749391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a2b4a-0b5b-496d-9eaa-2882a2b37095 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:13.759047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1249/2517] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:14.254375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:14.255792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:14.269135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8323e-09d9-4710-86c8-5be0752d3f41 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:14.274008 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:14.274645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:14.274922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:14.275194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] PATH_INFO: `/v3/projects/2d4c1a18f2c049169915a8f3fd8a57e7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:14.277289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4c1a18f2c049169915a8f3fd8a57e7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:14.279290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:14.409793 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4c1a18f2c049169915a8f3fd8a57e7', 'request_id': 'req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:14.504080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4c1a18f2c049169915a8f3fd8a57e7', 'request_id': 'req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:14.511115 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4c1a18f2c049169915a8f3fd8a57e7', 'request_id': 'req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:14.528558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] The token cache is being invalidate because project 2d4c1a18f2c049169915a8f3fd8a57e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:14.528558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:14.528558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1c6bc514-c3bc-4a60-b783-a4f36474ea7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:14.528558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1269/2518] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:14 2022] DELETE /identity/v3/projects/2d4c1a18f2c049169915a8f3fd8a57e7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:14.533246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e7fe8-6e70-4700-965d-625014801729 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:14.533708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e7fe8-6e70-4700-965d-625014801729 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:14.533859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e7fe8-6e70-4700-965d-625014801729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:14.554454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-973e7fe8-6e70-4700-965d-625014801729 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:14.555667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-973e7fe8-6e70-4700-965d-625014801729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:14.625994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1250/2519] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:14.640480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9035578-1ae5-4056-94ef-9d51119c2921 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:14.640781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9035578-1ae5-4056-94ef-9d51119c2921 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:14.641053 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9035578-1ae5-4056-94ef-9d51119c2921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:14.662705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a9035578-1ae5-4056-94ef-9d51119c2921 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:14.663936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a9035578-1ae5-4056-94ef-9d51119c2921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:14.672009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1270/2520] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:14.678667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e7fe8-6e70-4700-965d-625014801729 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:14.687746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e7fe8-6e70-4700-965d-625014801729 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:14.692725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:14.693413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:14.693803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:14.694155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:14.694932 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:14.695660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:14.697443 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:14.714349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1251/2521] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:14.715429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9035578-1ae5-4056-94ef-9d51119c2921 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:14.725376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9035578-1ae5-4056-94ef-9d51119c2921 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:14.730117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:14.730723 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:14.730994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:14.731272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:14.731911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:14.733852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:14.848977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1271/2522] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:14 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:14.853144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:14.863981 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bd5fc-8607-42cf-86b8-b81cb28832b3 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:14.923822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:14.924498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:14.924785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:14.925197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:14.925921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:14.927921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:15.056349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1252/2523] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:14 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:15.057699 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:15.069468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e4c8-9b03-48fe-9dd7-0a145fad357b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:15.074207 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:15.074815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:15.075099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:15.075373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:15.076021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:15.077679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:15.090484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1272/2524] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:15.094519 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:15.104942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea867e6-8b90-4ed4-a836-46c4641ca703 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:15.109591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed5c89-68f6-42d4-af25-ee25a1915f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:15.110266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed5c89-68f6-42d4-af25-ee25a1915f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:15.110806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed5c89-68f6-42d4-af25-ee25a1915f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:15.111079 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed5c89-68f6-42d4-af25-ee25a1915f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:15.111731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed5c89-68f6-42d4-af25-ee25a1915f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:15.113370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed5c89-68f6-42d4-af25-ee25a1915f20 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:15.125065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1253/2525] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:15.129601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:15.139669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825a2b5-0e53-4cc9-85dc-cdbb9b2ef747 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:15.144824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:15.145474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:15.145771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:15.146083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] PATH_INFO: `/v3/projects/74254b5f303f4ac5938ac07c7372e388/users/a39954537b904e20993b57e76f1c1e98/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:15.160706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74254b5f303f4ac5938ac07c7372e388, user_id=a39954537b904e20993b57e76f1c1e98, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:15.163372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:15.225325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1273/2526] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:15 2022] PUT /identity/v3/projects/74254b5f303f4ac5938ac07c7372e388/users/a39954537b904e20993b57e76f1c1e98/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:15.236547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:15.236831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:15.237134 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:15.267642 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] MFA Rules not processed for user `a39954537b904e20993b57e76f1c1e98`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:15.268859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:15.289687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:15.335621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1254/2527] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:15.346569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:15.348325 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:15.376678 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:15.409699 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847148e8-234d-4f40-871a-c22fcb0d33e1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:15.413492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99186a-2ac0-4e78-a853-558dc8d2b07f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:15.414139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99186a-2ac0-4e78-a853-558dc8d2b07f service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:15.414459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99186a-2ac0-4e78-a853-558dc8d2b07f service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:15.414731 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99186a-2ac0-4e78-a853-558dc8d2b07f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:15.423806 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99186a-2ac0-4e78-a853-558dc8d2b07f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:15.425311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99186a-2ac0-4e78-a853-558dc8d2b07f service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:15.434129 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1274/2528] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:16.471450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:16.472928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:16.537945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1514-a55e-4bc5-8500-b8c1df8e3843 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:16.542379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:16.543001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:16.543278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:16.543554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] PATH_INFO: `/v3/users/c32e36349ad947019eb2f531c61d981c` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:16.544797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32e36349ad947019eb2f531c61d981c)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:16.546772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:16.727021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32e36349ad947019eb2f531c61d981c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:16.727611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32e36349ad947019eb2f531c61d981c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:16.735039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32e36349ad947019eb2f531c61d981c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:16.747323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32e36349ad947019eb2f531c61d981c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:16.748063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32e36349ad947019eb2f531c61d981c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:16.850208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1255/2529] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:16 2022] DELETE /identity/v3/users/c32e36349ad947019eb2f531c61d981c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:16.854645 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:16.855154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:16.855292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:16.874658 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:16.876501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:16.936734 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1275/2530] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:16.947846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:16.949237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:16.995471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:17.007609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83c31-dfb2-43ee-ab84-e874c15530a1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:17.011170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:17.012068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:17.012068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:17.012374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:17.023628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:17.025451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:17.036464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1256/2531] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:17.494519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:17.496111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:17.512162 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f54a7-ba23-4171-ba26-868a69b5d745 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:17.514907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:17.515582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:17.516130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:17.516489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] PATH_INFO: `/v3/projects/d374f2c2ecbb4aaa84709f9ccff19bc8` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:17.517790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d374f2c2ecbb4aaa84709f9ccff19bc8)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:17.519726 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:17.647075 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd374f2c2ecbb4aaa84709f9ccff19bc8', 'request_id': 'req-f125a79f-e654-4d53-a585-a997ad9c8e12'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:17.755242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd374f2c2ecbb4aaa84709f9ccff19bc8', 'request_id': 'req-f125a79f-e654-4d53-a585-a997ad9c8e12'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:17.762388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd374f2c2ecbb4aaa84709f9ccff19bc8', 'request_id': 'req-f125a79f-e654-4d53-a585-a997ad9c8e12'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:17.773773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] The token cache is being invalidate because project d374f2c2ecbb4aaa84709f9ccff19bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:17.774057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:17.774706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:17.776195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1276/2532] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:17 2022] DELETE /identity/v3/projects/d374f2c2ecbb4aaa84709f9ccff19bc8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:17.780452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:17.791847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f46a2-f708-435f-a354-8e640e8af2a2 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:17.849607 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:17.850270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:17.850573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:17.851215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] PATH_INFO: `/v3/users/01c2a96fedb74e7e82d97ec40aabb42e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:17.852045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c2a96fedb74e7e82d97ec40aabb42e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:17.854047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:18.084446 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c2a96fedb74e7e82d97ec40aabb42e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.085130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c2a96fedb74e7e82d97ec40aabb42e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.092265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c2a96fedb74e7e82d97ec40aabb42e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.104745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c2a96fedb74e7e82d97ec40aabb42e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.105577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-54b72ebc-901c-49e2-bbea-bc37d812762a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c2a96fedb74e7e82d97ec40aabb42e'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.184283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1257/2533] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:17 2022] DELETE /identity/v3/users/01c2a96fedb74e7e82d97ec40aabb42e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:18.608143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:18.609588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:18.627991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125a79f-e654-4d53-a585-a997ad9c8e12 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:18.681082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:18.694552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:18.695317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:18.695568 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:18.695843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] PATH_INFO: `/v3/projects/868cf8d8e417423aa7143dd8c47551a5` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:18.697033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=868cf8d8e417423aa7143dd8c47551a5)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:18.698955 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:18.829588 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '868cf8d8e417423aa7143dd8c47551a5', 'request_id': 'req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.898078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '868cf8d8e417423aa7143dd8c47551a5', 'request_id': 'req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.904956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '868cf8d8e417423aa7143dd8c47551a5', 'request_id': 'req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.916026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] The token cache is being invalidate because project 868cf8d8e417423aa7143dd8c47551a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:18.917630 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.917630 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0b9019f9-94fd-4cc6-8b96-fbead8c7afa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:18.922001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1277/2534] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:18 2022] DELETE /identity/v3/projects/868cf8d8e417423aa7143dd8c47551a5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:18.924139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16242dfd-48f0-49e1-9b9a-e986a904d448 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:18.924595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16242dfd-48f0-49e1-9b9a-e986a904d448 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:18.924770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16242dfd-48f0-49e1-9b9a-e986a904d448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:18.945520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-16242dfd-48f0-49e1-9b9a-e986a904d448 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:18.946685 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-16242dfd-48f0-49e1-9b9a-e986a904d448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:19.008041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1258/2535] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:19.019254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302addff-26bd-4c5d-86d1-ae7e5057321f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.019532 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302addff-26bd-4c5d-86d1-ae7e5057321f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.019829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302addff-26bd-4c5d-86d1-ae7e5057321f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.046388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-302addff-26bd-4c5d-86d1-ae7e5057321f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:19.047705 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-302addff-26bd-4c5d-86d1-ae7e5057321f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:19.055854 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1278/2536] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:19.060068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16242dfd-48f0-49e1-9b9a-e986a904d448 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.066927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:19.070667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16242dfd-48f0-49e1-9b9a-e986a904d448 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:19.082094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:19.083980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.083980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.083980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.083980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:19.084440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:19.086092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:19.098729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1259/2537] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:19.103366 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302addff-26bd-4c5d-86d1-ae7e5057321f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.113443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302addff-26bd-4c5d-86d1-ae7e5057321f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:19.118123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:19.118727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.119004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.119301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.119929 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:19.121830 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:19.223376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1279/2538] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:19 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:19.224631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.236467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4060-d9da-4482-82a9-59170c1acbec admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:19.294010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:19.294613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.294886 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.295153 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.295771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:19.298475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:19.410548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1260/2539] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:19 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:19.414720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.427012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49aba1-5768-4f27-9751-930768f0d625 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:19.431704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:19.432343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.432653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.432932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.433617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:19.435239 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:19.446948 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1280/2540] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:19.451234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.462634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9c9ce-00e2-461c-afe8-ccd5bc3d927b admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:19.468295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28c144-cb9e-4f51-8d6a-de1907e93135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:19.475605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28c144-cb9e-4f51-8d6a-de1907e93135 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.475605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28c144-cb9e-4f51-8d6a-de1907e93135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.475605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28c144-cb9e-4f51-8d6a-de1907e93135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.475605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28c144-cb9e-4f51-8d6a-de1907e93135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:19.475605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28c144-cb9e-4f51-8d6a-de1907e93135 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:19.484227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1261/2541] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:19.488985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.499391 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07170f2-0df8-4417-b859-c83afab8ce5e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:19.504916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:19.505596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.505909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.506185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] PATH_INFO: `/v3/projects/dc4a4c3254a54c3cb30856a476d454f0/users/be4589cfecbc46a19dd3264597c10718/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.520971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc4a4c3254a54c3cb30856a476d454f0, user_id=be4589cfecbc46a19dd3264597c10718, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:19.523825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:19.621610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1281/2542] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:19 2022] PUT /identity/v3/projects/dc4a4c3254a54c3cb30856a476d454f0/users/be4589cfecbc46a19dd3264597c10718/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:19.636131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed2a0c-5792-421c-a7a7-fbf677c1dcb7 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.636437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed2a0c-5792-421c-a7a7-fbf677c1dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.637176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed2a0c-5792-421c-a7a7-fbf677c1dcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.674980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7aed2a0c-5792-421c-a7a7-fbf677c1dcb7 None None] MFA Rules not processed for user `be4589cfecbc46a19dd3264597c10718`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:19.676266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7aed2a0c-5792-421c-a7a7-fbf677c1dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:19.744680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1262/2543] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:19.749136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.828261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345bd5b-eacc-4a1f-8c10-6158ba1919a0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:19.833441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:19.834123 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:19.834413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:19.834693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:19.839055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:19.839055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:19.938808 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:19.946773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1282/2544] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:19 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:19.950841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed2a0c-5792-421c-a7a7-fbf677c1dcb7 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:19.962570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed2a0c-5792-421c-a7a7-fbf677c1dcb7 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:20.018820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:20.019368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.019648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.019910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.020547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:20.022542 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:20.030480 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:20.114810 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1263/2545] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:19 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:20.119000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:20.129342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed848f2-adc0-4eb8-97d1-60c1ba3d2b87 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:20.134774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:20.135444 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.135737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.136013 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.136689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:20.138383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:20.151226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1283/2546] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:20.155266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:20.166097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a8f55-7796-44d0-8039-94719277f0ef admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:20.170774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:20.171408 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.171687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.171958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] PATH_INFO: `/v3/projects/4e36a1a1d3fa489f9634024b86830830/users/e61ce0d844934a73a1fcbc9b8b3260e9/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.186513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e36a1a1d3fa489f9634024b86830830, user_id=e61ce0d844934a73a1fcbc9b8b3260e9, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:20.190337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:20.302151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1264/2547] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:20 2022] PUT /identity/v3/projects/4e36a1a1d3fa489f9634024b86830830/users/e61ce0d844934a73a1fcbc9b8b3260e9/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:20.306098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:20.316686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b44580-00c3-4be2-9fc5-378e3deaa083 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:20.379511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862404b6-8d04-46ba-938c-0c3cd151540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:20.380171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862404b6-8d04-46ba-938c-0c3cd151540e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.380478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862404b6-8d04-46ba-938c-0c3cd151540e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.381255 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862404b6-8d04-46ba-938c-0c3cd151540e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.381932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862404b6-8d04-46ba-938c-0c3cd151540e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:20.383521 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862404b6-8d04-46ba-938c-0c3cd151540e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:20.394892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1284/2548] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:20.399237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:20.411928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce925b-8a9b-4b45-90a6-1d036056c809 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:20.416640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:20.417302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.417590 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.417898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] PATH_INFO: `/v3/projects/4e36a1a1d3fa489f9634024b86830830/users/e61ce0d844934a73a1fcbc9b8b3260e9/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.419878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e36a1a1d3fa489f9634024b86830830, user_id=e61ce0d844934a73a1fcbc9b8b3260e9, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:20.422554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:20.433246 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1265/2549] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:20 2022] PUT /identity/v3/projects/4e36a1a1d3fa489f9634024b86830830/users/e61ce0d844934a73a1fcbc9b8b3260e9/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:20.450456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9ad2-633c-4ebd-8a23-5c8b80e41c87 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.450743 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9ad2-633c-4ebd-8a23-5c8b80e41c87 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.451042 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9ad2-633c-4ebd-8a23-5c8b80e41c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.484264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2f8d9ad2-633c-4ebd-8a23-5c8b80e41c87 None None] MFA Rules not processed for user `e61ce0d844934a73a1fcbc9b8b3260e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:20.485473 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2f8d9ad2-633c-4ebd-8a23-5c8b80e41c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:20.554918 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1285/2550] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:20.566632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:20.568291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:20.626780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf482e-53ea-4887-893c-147226620b94 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:20.631156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf07c2-32a8-4c01-99b9-1d8c5883aa61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:20.631768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf07c2-32a8-4c01-99b9-1d8c5883aa61 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.632042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf07c2-32a8-4c01-99b9-1d8c5883aa61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.632312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf07c2-32a8-4c01-99b9-1d8c5883aa61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.641707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf07c2-32a8-4c01-99b9-1d8c5883aa61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:20.643164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf07c2-32a8-4c01-99b9-1d8c5883aa61 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:20.653449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1266/2551] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:20.682390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d9ad2-633c-4ebd-8a23-5c8b80e41c87 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:20.747114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d9ad2-633c-4ebd-8a23-5c8b80e41c87 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:20.751657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:20.752328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:20.752610 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:20.752997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] PATH_INFO: `/v3/users/be4589cfecbc46a19dd3264597c10718` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:20.754163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4589cfecbc46a19dd3264597c10718)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:20.756037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:20.769841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:20.969550 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4589cfecbc46a19dd3264597c10718'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:20.970302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4589cfecbc46a19dd3264597c10718'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:20.977923 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4589cfecbc46a19dd3264597c10718'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:20.990337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4589cfecbc46a19dd3264597c10718'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:20.990996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4589cfecbc46a19dd3264597c10718'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:21.046516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1286/2552] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:20 2022] DELETE /identity/v3/users/be4589cfecbc46a19dd3264597c10718 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:21.050972 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:21.051433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:21.052169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:21.071084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:21.072323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:21.138340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1267/2553] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:21.149213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:21.150665 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:21.209279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff53a7-d668-418f-bae6-6c0dc9a99122 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:21.212901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:21.213503 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:21.213816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:21.214092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:21.223722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:21.225240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:21.235404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1287/2554] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:21.651961 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:21.653382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:21.667406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13003f-06e6-4520-94d5-7c6fd0c91940 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:21.672367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:21.673000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:21.673277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:21.673547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] PATH_INFO: `/v3/projects/dc4a4c3254a54c3cb30856a476d454f0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:21.674793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc4a4c3254a54c3cb30856a476d454f0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:21.676687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:21.778728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc4a4c3254a54c3cb30856a476d454f0', 'request_id': 'req-0e72470a-1c24-4350-b366-01789129e7fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:21.780364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:21.867317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc4a4c3254a54c3cb30856a476d454f0', 'request_id': 'req-0e72470a-1c24-4350-b366-01789129e7fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:21.875737 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc4a4c3254a54c3cb30856a476d454f0', 'request_id': 'req-0e72470a-1c24-4350-b366-01789129e7fa'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:21.887035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] The token cache is being invalidate because project dc4a4c3254a54c3cb30856a476d454f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:21.887416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:21.888108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:21.889617 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1268/2555] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:21 2022] DELETE /identity/v3/projects/dc4a4c3254a54c3cb30856a476d454f0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:21.894387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:21.911740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163c750d-bb17-4a89-90f9-535df07a79ea service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:21.968599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:21.969225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:21.969501 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:21.969802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] PATH_INFO: `/v3/users/e61ce0d844934a73a1fcbc9b8b3260e9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:21.970999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61ce0d844934a73a1fcbc9b8b3260e9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:21.972980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:22.136657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:22.148797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61ce0d844934a73a1fcbc9b8b3260e9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.149461 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61ce0d844934a73a1fcbc9b8b3260e9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.156576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61ce0d844934a73a1fcbc9b8b3260e9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.168780 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61ce0d844934a73a1fcbc9b8b3260e9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.169429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-44558640-4b7b-4a16-a13c-622023dac7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61ce0d844934a73a1fcbc9b8b3260e9'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.258410 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1288/2556] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:21 2022] DELETE /identity/v3/users/e61ce0d844934a73a1fcbc9b8b3260e9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:22.719702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:22.721151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:22.734941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72470a-1c24-4350-b366-01789129e7fa admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:22.792072 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:22.792704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:22.792982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:22.793253 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] PATH_INFO: `/v3/projects/4e36a1a1d3fa489f9634024b86830830` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:22.794349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e36a1a1d3fa489f9634024b86830830)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:22.796276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:22.898425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e36a1a1d3fa489f9634024b86830830', 'request_id': 'req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.960935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e36a1a1d3fa489f9634024b86830830', 'request_id': 'req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.968350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e36a1a1d3fa489f9634024b86830830', 'request_id': 'req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.979768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] The token cache is being invalidate because project 4e36a1a1d3fa489f9634024b86830830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:22.980055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.980704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5a35d8d5-196d-4e5a-8631-9c6545ccfad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:22.982213 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1269/2557] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:22 2022] DELETE /identity/v3/projects/4e36a1a1d3fa489f9634024b86830830 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:22.987088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46776-2612-42bb-8d9a-ad521ea57b5e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:22.987609 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46776-2612-42bb-8d9a-ad521ea57b5e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:22.987707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46776-2612-42bb-8d9a-ad521ea57b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.007389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-98e46776-2612-42bb-8d9a-ad521ea57b5e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:23.008621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-98e46776-2612-42bb-8d9a-ad521ea57b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:23.068041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1289/2558] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:23.075923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79e843-7f9a-48b4-b52b-595feec9d747 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.076177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79e843-7f9a-48b4-b52b-595feec9d747 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.076454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79e843-7f9a-48b4-b52b-595feec9d747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.098552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4e79e843-7f9a-48b4-b52b-595feec9d747 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:23.099855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4e79e843-7f9a-48b4-b52b-595feec9d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:23.108679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1270/2559] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:23.113533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46776-2612-42bb-8d9a-ad521ea57b5e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.123490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46776-2612-42bb-8d9a-ad521ea57b5e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:23.128269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:23.130303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.130303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.130303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.130303 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:23.130863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:23.132435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:23.145708 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1290/2560] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:23.149864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79e843-7f9a-48b4-b52b-595feec9d747 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.159898 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79e843-7f9a-48b4-b52b-595feec9d747 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:23.164546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:23.165138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.165441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.165714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.166374 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:23.168266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:23.259824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1271/2561] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:23 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:23.263767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.274788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf148-73db-4128-96c3-98cfd737827d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:23.332909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:23.333694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.334010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.334286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.334927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:23.336840 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:23.444857 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1291/2562] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:23 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:23.448788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.459762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeae81-b563-401c-8d3a-acab055ca6dd admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:23.464288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185856c-e743-4133-8876-7581046be49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:23.464882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185856c-e743-4133-8876-7581046be49f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.465269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185856c-e743-4133-8876-7581046be49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.465571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185856c-e743-4133-8876-7581046be49f admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.466241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185856c-e743-4133-8876-7581046be49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:23.467813 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185856c-e743-4133-8876-7581046be49f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:23.478010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1272/2563] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:23.481707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.491952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9609df-8c04-427c-9a9a-63553f273ea7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:23.496689 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cea7f-df89-47eb-b5ec-f21c77da68bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:23.497599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cea7f-df89-47eb-b5ec-f21c77da68bb admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.497940 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cea7f-df89-47eb-b5ec-f21c77da68bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.498210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cea7f-df89-47eb-b5ec-f21c77da68bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.498847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cea7f-df89-47eb-b5ec-f21c77da68bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:23.501006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cea7f-df89-47eb-b5ec-f21c77da68bb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:23.512236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1292/2564] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:23.516275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185856c-e743-4133-8876-7581046be49f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.526919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185856c-e743-4133-8876-7581046be49f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:23.531681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:23.532322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.532857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.533137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] PATH_INFO: `/v3/projects/399804bb79b143b6b586ef21812a77ec/users/f9d328d51df4493397fec24c2ba191fe/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.548399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399804bb79b143b6b586ef21812a77ec, user_id=f9d328d51df4493397fec24c2ba191fe, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:23.551566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:23.641674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1273/2565] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:23 2022] PUT /identity/v3/projects/399804bb79b143b6b586ef21812a77ec/users/f9d328d51df4493397fec24c2ba191fe/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:23.653831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb90065-5044-4a3d-a011-f1ed6574e1f9 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.654126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb90065-5044-4a3d-a011-f1ed6574e1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.654428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb90065-5044-4a3d-a011-f1ed6574e1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.683450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bfb90065-5044-4a3d-a011-f1ed6574e1f9 None None] MFA Rules not processed for user `f9d328d51df4493397fec24c2ba191fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:23.684648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bfb90065-5044-4a3d-a011-f1ed6574e1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:23.688500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:23.749214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1293/2566] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:23.754265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.759842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:23.821918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f547be0-2f81-42ea-9acb-ee2ae4174011 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:23.826583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:23.827180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:23.827461 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:23.827731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:23.828367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:23.830298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:23.937579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1274/2567] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:23 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:23.942543 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb90065-5044-4a3d-a011-f1ed6574e1f9 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:23.953100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb90065-5044-4a3d-a011-f1ed6574e1f9 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:24.136755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:24.137438 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.137635 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.137943 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.138579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:24.140477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:24.247585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1294/2568] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:23 2022] POST /identity/v3/users => generated 300 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:24.248827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:24.259042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bd983-eb5c-46a2-bcab-871167e11606 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:24.263579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:24.264177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.264457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.264746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.265383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:24.267052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:24.277478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1275/2569] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:24.281496 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:24.292194 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99743d7a-df0b-44e4-bcd9-a13d46b98774 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:24.297181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:24.297868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.298153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.298428 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] PATH_INFO: `/v3/projects/a2e845710ba94d7b8af64f1229cd833e/users/9d0a6c2e56dd4a05b961adc5b2bdff5b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.314784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e845710ba94d7b8af64f1229cd833e, user_id=9d0a6c2e56dd4a05b961adc5b2bdff5b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:24.317406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:24.417467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1295/2570] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:24 2022] PUT /identity/v3/projects/a2e845710ba94d7b8af64f1229cd833e/users/9d0a6c2e56dd4a05b961adc5b2bdff5b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:24.421623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:24.434819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552cea2-27ae-44a8-bfb4-2f60c286970c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:24.510502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b6e00-8bd3-4519-8d67-793981a15c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:24.511094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b6e00-8bd3-4519-8d67-793981a15c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.511379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b6e00-8bd3-4519-8d67-793981a15c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.511745 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b6e00-8bd3-4519-8d67-793981a15c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.513163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b6e00-8bd3-4519-8d67-793981a15c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:24.514015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b6e00-8bd3-4519-8d67-793981a15c88 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:24.528752 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1276/2571] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:24.530029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:24.541669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bfdc5-92a2-4a21-9f3b-dd0642254d8e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:24.546432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:24.547093 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.547401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.547679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] PATH_INFO: `/v3/projects/a2e845710ba94d7b8af64f1229cd833e/users/9d0a6c2e56dd4a05b961adc5b2bdff5b/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.549663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e845710ba94d7b8af64f1229cd833e, user_id=9d0a6c2e56dd4a05b961adc5b2bdff5b, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:24.552662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:24.563832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1296/2572] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:24 2022] PUT /identity/v3/projects/a2e845710ba94d7b8af64f1229cd833e/users/9d0a6c2e56dd4a05b961adc5b2bdff5b/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:24.574639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9f1f-79fe-4860-a183-1c80f283672c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.575025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9f1f-79fe-4860-a183-1c80f283672c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.575370 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9f1f-79fe-4860-a183-1c80f283672c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.607593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-eb5d9f1f-79fe-4860-a183-1c80f283672c None None] MFA Rules not processed for user `9d0a6c2e56dd4a05b961adc5b2bdff5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:24.608844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-eb5d9f1f-79fe-4860-a183-1c80f283672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:24.677063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:24.680613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1277/2573] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:24.692674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:24.694430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:24.751254 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80740c6a-539a-4348-8cbd-fca827a8e573 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:24.755822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627f8d1-5842-4328-8942-e6f99e6cf62e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:24.756425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f8d1-5842-4328-8942-e6f99e6cf62e service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.756707 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f8d1-5842-4328-8942-e6f99e6cf62e service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.756984 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627f8d1-5842-4328-8942-e6f99e6cf62e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.765928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f8d1-5842-4328-8942-e6f99e6cf62e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:24.767341 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627f8d1-5842-4328-8942-e6f99e6cf62e service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:24.773261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:24.778070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1297/2574] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:24.845455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d9f1f-79fe-4860-a183-1c80f283672c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:24.909586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d9f1f-79fe-4860-a183-1c80f283672c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:24.914156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:24.914779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:24.915053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:24.915321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] PATH_INFO: `/v3/users/f9d328d51df4493397fec24c2ba191fe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:24.916435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9d328d51df4493397fec24c2ba191fe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:24.918448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:25.090511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9d328d51df4493397fec24c2ba191fe'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.091165 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9d328d51df4493397fec24c2ba191fe'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.098274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9d328d51df4493397fec24c2ba191fe'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.110141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9d328d51df4493397fec24c2ba191fe'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.110797 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9d328d51df4493397fec24c2ba191fe'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.223625 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1278/2575] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:24 2022] DELETE /identity/v3/users/f9d328d51df4493397fec24c2ba191fe => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:25.228126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:25.228604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:25.228700 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:25.247327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:25.248510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:25.306327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1298/2576] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:25.316567 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:25.318012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:25.375314 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bab7f2-32cd-4895-afa2-5f6fca316600 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:25.380033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f584fd4-73ac-492f-934f-e4026b5ed740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:25.380649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f584fd4-73ac-492f-934f-e4026b5ed740 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:25.380930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f584fd4-73ac-492f-934f-e4026b5ed740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:25.381267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f584fd4-73ac-492f-934f-e4026b5ed740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:25.390928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f584fd4-73ac-492f-934f-e4026b5ed740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:25.392372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f584fd4-73ac-492f-934f-e4026b5ed740 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:25.402588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1279/2577] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:25.501452 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:25.503627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:25.515404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154ad47-dada-47dd-8381-b20f2205afd4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:25.520315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:25.520942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:25.521222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:25.521498 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] PATH_INFO: `/v3/users/a39954537b904e20993b57e76f1c1e98` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:25.522681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39954537b904e20993b57e76f1c1e98)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:25.524654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:25.744552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39954537b904e20993b57e76f1c1e98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.746339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39954537b904e20993b57e76f1c1e98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.754530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39954537b904e20993b57e76f1c1e98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.768716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39954537b904e20993b57e76f1c1e98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.770101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39954537b904e20993b57e76f1c1e98'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:25.916071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1299/2578] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:25 2022] DELETE /identity/v3/users/a39954537b904e20993b57e76f1c1e98 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:25.920677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:25.921233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:25.921233 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:25.930643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:25.932323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:25.941131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:25.942298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:25.962796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:26.013640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef065-6c99-4380-91ab-11cace75198e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:26.019391 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1280/2579] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:26.022390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:26.023046 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:26.023389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:26.023667 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] PATH_INFO: `/v3/projects/399804bb79b143b6b586ef21812a77ec` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:26.025067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399804bb79b143b6b586ef21812a77ec)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:26.027061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:26.027359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:26.029164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:26.057578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:26.091968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdced9-2661-4677-b6bc-b49206ff453b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:26.098334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:26.098796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:26.099080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:26.099351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:26.110342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:26.111816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:26.122017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1281/2580] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:26.166863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399804bb79b143b6b586ef21812a77ec', 'request_id': 'req-9a54bb87-f20c-442d-8d14-29e2cb1e6863'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.241212 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399804bb79b143b6b586ef21812a77ec', 'request_id': 'req-9a54bb87-f20c-442d-8d14-29e2cb1e6863'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.256056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399804bb79b143b6b586ef21812a77ec', 'request_id': 'req-9a54bb87-f20c-442d-8d14-29e2cb1e6863'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.278528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] The token cache is being invalidate because project 399804bb79b143b6b586ef21812a77ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:26.279204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.280159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.282355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1300/2581] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:25 2022] DELETE /identity/v3/projects/399804bb79b143b6b586ef21812a77ec => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:26.287160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:26.297600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5cb24-44ca-4d7a-93e8-aaf52246c67e service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:26.379477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:26.380395 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:26.380744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:26.381087 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] PATH_INFO: `/v3/users/9d0a6c2e56dd4a05b961adc5b2bdff5b` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:26.382422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0a6c2e56dd4a05b961adc5b2bdff5b)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:26.384630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:26.654004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0a6c2e56dd4a05b961adc5b2bdff5b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.656731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0a6c2e56dd4a05b961adc5b2bdff5b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.668175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0a6c2e56dd4a05b961adc5b2bdff5b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.678871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:26.683652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0a6c2e56dd4a05b961adc5b2bdff5b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.685318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8db24975-0703-405d-9356-c06c54b1facf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0a6c2e56dd4a05b961adc5b2bdff5b'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:26.786778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1282/2582] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:26 2022] DELETE /identity/v3/users/9d0a6c2e56dd4a05b961adc5b2bdff5b => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:26.871787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:26.873226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:26.900098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54bb87-f20c-442d-8d14-29e2cb1e6863 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:26.975061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:26.975711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:26.975993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:26.978126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] PATH_INFO: `/v3/projects/74254b5f303f4ac5938ac07c7372e388` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:26.979911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=74254b5f303f4ac5938ac07c7372e388)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:26.982253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.162775 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74254b5f303f4ac5938ac07c7372e388', 'request_id': 'req-392a6cbe-28a8-470c-b122-1b702c2e65ee'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.285188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74254b5f303f4ac5938ac07c7372e388', 'request_id': 'req-392a6cbe-28a8-470c-b122-1b702c2e65ee'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.292912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74254b5f303f4ac5938ac07c7372e388', 'request_id': 'req-392a6cbe-28a8-470c-b122-1b702c2e65ee'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.305668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] The token cache is being invalidate because project 74254b5f303f4ac5938ac07c7372e388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:27.306296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.307563 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-392a6cbe-28a8-470c-b122-1b702c2e65ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.309545 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1301/2583] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:26 2022] DELETE /identity/v3/projects/74254b5f303f4ac5938ac07c7372e388 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:27.314505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.315036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.315261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.336824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:27.338458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:27.405711 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1283/2584] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:27.414233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58982-6e8b-4956-9d18-3a1ac8ec1ca1 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.414788 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58982-6e8b-4956-9d18-3a1ac8ec1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.418584 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa58982-6e8b-4956-9d18-3a1ac8ec1ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.422885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.427684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:27.451154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5aa58982-6e8b-4956-9d18-3a1ac8ec1ca1 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:27.453572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a04949-9245-47cd-a6e4-4b13d99a9b22 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.458518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5aa58982-6e8b-4956-9d18-3a1ac8ec1ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:27.459825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.460634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.460986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.461369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] PATH_INFO: `/v3/projects/a2e845710ba94d7b8af64f1229cd833e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.462816 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e845710ba94d7b8af64f1229cd833e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.464824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.476552 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1302/2585] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:27.481710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa58982-6e8b-4956-9d18-3a1ac8ec1ca1 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.493261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa58982-6e8b-4956-9d18-3a1ac8ec1ca1 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.497946 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.499202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.499474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.499913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.500786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:27.501014 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.503340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.522403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1303/2586] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:27.523564 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.535695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829a2e5-140a-40f1-a9f8-eba4e8eddbed admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.540458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.541066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.541340 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.541608 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.542275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.544140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.557443 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:27.607011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e845710ba94d7b8af64f1229cd833e', 'request_id': 'req-daee19a2-4b48-428f-8574-32a12d7f3b87'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.663197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1304/2587] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:27 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:27.667337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.680958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e845710ba94d7b8af64f1229cd833e', 'request_id': 'req-daee19a2-4b48-428f-8574-32a12d7f3b87'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.690502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e845710ba94d7b8af64f1229cd833e', 'request_id': 'req-daee19a2-4b48-428f-8574-32a12d7f3b87'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.700606 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] The token cache is being invalidate because project a2e845710ba94d7b8af64f1229cd833e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:27.700906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.701559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-daee19a2-4b48-428f-8574-32a12d7f3b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:27.703160 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1284/2588] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:27 2022] DELETE /identity/v3/projects/a2e845710ba94d7b8af64f1229cd833e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:27.708588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efcac-e9d9-427c-800a-184b82f213c9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.708943 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efcac-e9d9-427c-800a-184b82f213c9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.709267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6efcac-e9d9-427c-800a-184b82f213c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.729481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-8c6efcac-e9d9-427c-800a-184b82f213c9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:27.730783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8c6efcac-e9d9-427c-800a-184b82f213c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:27.746314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75a60-e04c-4b15-af89-4ee4089102c8 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.751260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.751870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.752149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.752419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.753802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.755690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.799478 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1285/2589] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:27.807104 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1a05-f7bb-4d40-9fbc-c361da841967 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.820317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1a05-f7bb-4d40-9fbc-c361da841967 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.820317 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea1a05-f7bb-4d40-9fbc-c361da841967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.827363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9fea1a05-f7bb-4d40-9fbc-c361da841967 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:27.828593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9fea1a05-f7bb-4d40-9fbc-c361da841967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:27.838918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1286/2590] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:27.840136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1305/2591] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:27 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:27.843450 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea1a05-f7bb-4d40-9fbc-c361da841967 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.844050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.846488 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:27.850629 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:27.855901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea1a05-f7bb-4d40-9fbc-c361da841967 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.860899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7100af1-a475-4872-ad90-b9fb362c8b36 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.863518 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.866162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.866162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.866162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.866162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:27.866641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.866751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.866852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.866852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.866852 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.867378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.867491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.868972 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.880176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1306/2592] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:27.884329 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.885686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1287/2593] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:27.888910 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.897933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa005a7-0dcb-428b-a574-528a98a1de12 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.898242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ee915-f94d-4d84-aefd-9b2feaa59f40 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.903728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.904383 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.904609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.904916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.906423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.908143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.908633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.909232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.909544 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.909861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:27.910536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:27.912919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:27.927687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1307/2594] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:27.931407 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:27.943493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda639-b582-4841-97c7-695106318999 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:27.948659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd803ef-730a-4a23-b187-162a6c5be05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:27.949314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd803ef-730a-4a23-b187-162a6c5be05d admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:27.949592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd803ef-730a-4a23-b187-162a6c5be05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:27.949898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd803ef-730a-4a23-b187-162a6c5be05d admin admin] PATH_INFO: `/v3/projects/4f438580a60c44a69f157ef64800085c/users/60f0276f7f714af194dfc4dec5845b4c/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.004200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd803ef-730a-4a23-b187-162a6c5be05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f438580a60c44a69f157ef64800085c, user_id=60f0276f7f714af194dfc4dec5845b4c, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.004200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd803ef-730a-4a23-b187-162a6c5be05d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.228251 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1288/2595] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:27 2022] POST /identity/v3/projects => generated 381 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:28.232382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.255460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1308/2596] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:27 2022] PUT /identity/v3/projects/4f438580a60c44a69f157ef64800085c/users/60f0276f7f714af194dfc4dec5845b4c/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:28.267362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0b36-9cd6-41fc-8413-5bbfdcac3e86 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.267760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0b36-9cd6-41fc-8413-5bbfdcac3e86 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.268059 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b0b36-9cd6-41fc-8413-5bbfdcac3e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.297698 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-572b0b36-9cd6-41fc-8413-5bbfdcac3e86 None None] MFA Rules not processed for user `60f0276f7f714af194dfc4dec5845b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:28.299120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-572b0b36-9cd6-41fc-8413-5bbfdcac3e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:28.306092 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbee203-5f0a-433c-b91e-2c9b0513e689 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.380846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.380846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.380846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.380846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.381679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.383678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.386509 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1309/2597] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:28.391215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b0b36-9cd6-41fc-8413-5bbfdcac3e86 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.403750 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b0b36-9cd6-41fc-8413-5bbfdcac3e86 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.408919 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.409597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.409974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.410317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.411056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.413005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.505192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1289/2598] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:28 2022] POST /identity/v3/users => generated 291 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:28.506720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1310/2599] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:28 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:28.509008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.509402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.519901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f88f5-0e2c-4e00-969e-e4c400877831 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.573174 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9153c-73a2-40aa-83dd-9115b0d0f86d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.577805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.578009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.578435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.578574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.578717 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.578835 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.578993 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.579150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.579639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.579756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.581432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.581646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.592040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1290/2600] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:28.595858 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.606151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02444e25-d547-4abf-b136-542421c3d28d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.610762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.611334 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.611610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.611947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.613234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.614848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.628477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1291/2601] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:28.632556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.642227 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff41ba-cf9a-40f6-bd33-addac1187bea admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.648243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.648899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.649174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.649473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] PATH_INFO: `/v3/projects/a4b1c18c5a924947b343a2bdef982012/users/35a923c28abd4902aadbcffdd8670824/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.664814 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b1c18c5a924947b343a2bdef982012, user_id=35a923c28abd4902aadbcffdd8670824, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.667483 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.712309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1311/2602] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:28 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:28.716153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.726107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fca2f9-4178-469e-a63a-32484077f74b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.730817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52161f65-14dd-4a7c-9703-2ce680a8b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.731386 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52161f65-14dd-4a7c-9703-2ce680a8b481 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.732250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52161f65-14dd-4a7c-9703-2ce680a8b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.732533 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52161f65-14dd-4a7c-9703-2ce680a8b481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.733166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52161f65-14dd-4a7c-9703-2ce680a8b481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.734810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52161f65-14dd-4a7c-9703-2ce680a8b481 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.742026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1292/2603] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:28 2022] PUT /identity/v3/projects/a4b1c18c5a924947b343a2bdef982012/users/35a923c28abd4902aadbcffdd8670824/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:28.746381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1312/2604] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:28.750381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.756470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd735-85c5-43ae-804c-be77e1392894 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.756711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd735-85c5-43ae-804c-be77e1392894 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.757006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd735-85c5-43ae-804c-be77e1392894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.785629 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-549cd735-85c5-43ae-804c-be77e1392894 None None] MFA Rules not processed for user `35a923c28abd4902aadbcffdd8670824`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:28.790103 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-549cd735-85c5-43ae-804c-be77e1392894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:28.812771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7b528-6612-478e-9201-afd15d3d97e7 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.817553 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.824646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.824646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.824646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] PATH_INFO: `/v3/projects/e71b94d6d9b249eebf347f32935c91d7/users/5803a5d5852d44de817b89ba80e676b4/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.833586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71b94d6d9b249eebf347f32935c91d7, user_id=5803a5d5852d44de817b89ba80e676b4, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.836447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.851175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1313/2605] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:28.855606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd735-85c5-43ae-804c-be77e1392894 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:28.876043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd735-85c5-43ae-804c-be77e1392894 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:28.876043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:28.876043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:28.876043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:28.876467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:28.876467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:28.876467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:28.992994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1293/2606] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:28 2022] PUT /identity/v3/projects/e71b94d6d9b249eebf347f32935c91d7/users/5803a5d5852d44de817b89ba80e676b4/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:28.996914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.022193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1314/2607] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:28 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:29.027773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.061706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f2ee2-12db-4723-953e-38790274bede admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.076432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:29.089885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225dc4a-51a4-43ca-83ad-408ae9e09fe7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.094738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.095345 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.095627 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.095903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.096736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.098784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.117684 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.126908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.126908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.126908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.126908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.126908 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.131609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1294/2608] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:29.135724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.145977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304f3b-acda-43c1-bd5e-d3437f2b3312 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.150600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f9864-f0f3-433a-9093-df6390334bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.151236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9864-f0f3-433a-9093-df6390334bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.151512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9864-f0f3-433a-9093-df6390334bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.151782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9864-f0f3-433a-9093-df6390334bc4 admin admin] PATH_INFO: `/v3/projects/e71b94d6d9b249eebf347f32935c91d7/users/5803a5d5852d44de817b89ba80e676b4/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.153708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f9864-f0f3-433a-9093-df6390334bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71b94d6d9b249eebf347f32935c91d7, user_id=5803a5d5852d44de817b89ba80e676b4, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.157841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f9864-f0f3-433a-9093-df6390334bc4 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.166958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1295/2609] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:29 2022] PUT /identity/v3/projects/e71b94d6d9b249eebf347f32935c91d7/users/5803a5d5852d44de817b89ba80e676b4/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:29.177570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.177893 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.178168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.206283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] MFA Rules not processed for user `5803a5d5852d44de817b89ba80e676b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:29.207489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:29.212597 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1315/2610] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:29 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:29.214434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.224893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdba9d-2d00-4668-bcb5-e0b40109a4de admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.287671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1296/2611] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:29.290016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.290749 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.291036 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.291313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.291964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.295786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.299209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.301107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:29.308539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1316/2612] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:29.315864 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.325687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696594f6-ed31-46da-b4ae-1a7af020c8a1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.331733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.332376 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.332818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.333098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] PATH_INFO: `/v3/projects/51eece68ed044614a63e101672ea8372/users/b541be5ac0e84f369422c546e7ddbf83/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.343248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:29.351489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51eece68ed044614a63e101672ea8372, user_id=b541be5ac0e84f369422c546e7ddbf83, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.354998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.382423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a870a-b026-4acd-9904-e143c9a18d24 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.382423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.382423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.382423 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.383194 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.390994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.392419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.402445 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1297/2613] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:29.432650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:29.458551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1317/2614] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:29 2022] PUT /identity/v3/projects/51eece68ed044614a63e101672ea8372/users/b541be5ac0e84f369422c546e7ddbf83/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:29.466989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.478427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0eb9-ea40-4904-971e-dcf5b0e7e7b3 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.539582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f9110-adaf-4801-b803-e77351d828ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.540188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9110-adaf-4801-b803-e77351d828ac admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.540465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9110-adaf-4801-b803-e77351d828ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.540735 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9110-adaf-4801-b803-e77351d828ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.541365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f9110-adaf-4801-b803-e77351d828ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.542980 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f9110-adaf-4801-b803-e77351d828ac admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.553611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1298/2615] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:29.558096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.583311 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5a8f1-0a65-45fe-902a-7f9198716708 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.588367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.589249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.589686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.590139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] PATH_INFO: `/v3/projects/51eece68ed044614a63e101672ea8372/users/b541be5ac0e84f369422c546e7ddbf83/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.592913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51eece68ed044614a63e101672ea8372, user_id=b541be5ac0e84f369422c546e7ddbf83, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.595569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.607111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1318/2616] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:29 2022] PUT /identity/v3/projects/51eece68ed044614a63e101672ea8372/users/b541be5ac0e84f369422c546e7ddbf83/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:29.618287 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767c98f-f0de-4fd3-a9f6-c860d0059f8f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.618571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767c98f-f0de-4fd3-a9f6-c860d0059f8f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.618845 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767c98f-f0de-4fd3-a9f6-c860d0059f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.652785 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f767c98f-f0de-4fd3-a9f6-c860d0059f8f None None] MFA Rules not processed for user `b541be5ac0e84f369422c546e7ddbf83`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:29.654144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f767c98f-f0de-4fd3-a9f6-c860d0059f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:29.729272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:29.734991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1299/2617] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:29.746617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:29.748358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:29.823001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8ef50-6fdb-4e71-b40c-8104cea1c537 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:29.827045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:29.827657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:29.828028 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:29.828302 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:29.841499 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:29.846272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:29.856005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1319/2618] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:30.063477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:30.655240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767c98f-f0de-4fd3-a9f6-c860d0059f8f None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:30.719610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767c98f-f0de-4fd3-a9f6-c860d0059f8f None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:30.724743 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:30.725382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:30.725661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:30.725975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] PATH_INFO: `/v3/users/60f0276f7f714af194dfc4dec5845b4c` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:30.727128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f0276f7f714af194dfc4dec5845b4c)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:30.729058 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:30.741909 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:30.753038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1e455-1670-4222-9f84-4fac58b6f3e2 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:30.758493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:30.759159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:30.759441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:30.759713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] PATH_INFO: `/v3/users/35a923c28abd4902aadbcffdd8670824` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:30.760829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a923c28abd4902aadbcffdd8670824)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:30.762759 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:30.948701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f0276f7f714af194dfc4dec5845b4c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:30.949364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f0276f7f714af194dfc4dec5845b4c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:30.958433 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f0276f7f714af194dfc4dec5845b4c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:30.970185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f0276f7f714af194dfc4dec5845b4c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:30.970872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-167165cf-ba86-4bf4-bf4d-2e6275397fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f0276f7f714af194dfc4dec5845b4c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:31.006248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:31.015441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a923c28abd4902aadbcffdd8670824'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:31.016170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a923c28abd4902aadbcffdd8670824'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:31.023484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a923c28abd4902aadbcffdd8670824'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:31.035639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a923c28abd4902aadbcffdd8670824'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:31.036309 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a923c28abd4902aadbcffdd8670824'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:31.077421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1300/2619] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:30 2022] DELETE /identity/v3/users/60f0276f7f714af194dfc4dec5845b4c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:31.081766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4703bcd-5b77-40e5-b47b-39349c25c7cb None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:31.082061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4703bcd-5b77-40e5-b47b-39349c25c7cb None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:31.082335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4703bcd-5b77-40e5-b47b-39349c25c7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:31.094892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:31.101393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e4703bcd-5b77-40e5-b47b-39349c25c7cb None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:31.102681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e4703bcd-5b77-40e5-b47b-39349c25c7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:31.162196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1301/2620] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:31.166413 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1320/2621] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:30 2022] DELETE /identity/v3/users/35a923c28abd4902aadbcffdd8670824 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:31.170677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53b02-7b2f-486f-b39a-8c0ca201950b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:31.170815 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:31.170971 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53b02-7b2f-486f-b39a-8c0ca201950b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:31.171372 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53b02-7b2f-486f-b39a-8c0ca201950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:31.172114 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:31.195630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f1e53b02-7b2f-486f-b39a-8c0ca201950b None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:31.196838 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f1e53b02-7b2f-486f-b39a-8c0ca201950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:31.231741 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063421cd-5f1d-4f0f-bac8-ead4691ec08a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:31.235396 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:31.235995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:31.236284 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:31.236569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:31.258243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1302/2622] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:31.267845 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53b02-7b2f-486f-b39a-8c0ca201950b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:31.277610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53b02-7b2f-486f-b39a-8c0ca201950b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:31.282603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:31.283402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:31.283885 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:31.284173 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:31.293473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:31.295698 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:31.299167 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:31.301096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:31.306715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1303/2623] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:31.310901 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1321/2624] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:31.919811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:31.921082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:31.921556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:31.923377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:31.937266 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d3bc-60f9-4c65-be09-74099022b52f service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:31.948016 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:31.948713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:31.948996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:31.949270 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] PATH_INFO: `/v3/projects/a4b1c18c5a924947b343a2bdef982012` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:31.950404 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b1c18c5a924947b343a2bdef982012)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:31.951730 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca240019-888c-409f-94cc-d68a889059ad service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:31.952291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:31.957667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:31.958426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:31.958855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:31.959291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] PATH_INFO: `/v3/projects/4f438580a60c44a69f157ef64800085c` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:31.960758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f438580a60c44a69f157ef64800085c)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:31.964337 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:32.093655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b1c18c5a924947b343a2bdef982012', 'request_id': 'req-d7002493-ada1-4244-9a68-dfcf7dc44754'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.094033 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f438580a60c44a69f157ef64800085c', 'request_id': 'req-e02f9c13-c794-4756-ae42-59f18fb69a19'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.161032 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f438580a60c44a69f157ef64800085c', 'request_id': 'req-e02f9c13-c794-4756-ae42-59f18fb69a19'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.168696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f438580a60c44a69f157ef64800085c', 'request_id': 'req-e02f9c13-c794-4756-ae42-59f18fb69a19'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.182729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] The token cache is being invalidate because project 4f438580a60c44a69f157ef64800085c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:32.183170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.183992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.185966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1304/2625] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:31 2022] DELETE /identity/v3/projects/4f438580a60c44a69f157ef64800085c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:32.189788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:32.191799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:32.211442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:32.221106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b1c18c5a924947b343a2bdef982012', 'request_id': 'req-d7002493-ada1-4244-9a68-dfcf7dc44754'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.222621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:32.228774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b1c18c5a924947b343a2bdef982012', 'request_id': 'req-d7002493-ada1-4244-9a68-dfcf7dc44754'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.239736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] The token cache is being invalidate because project a4b1c18c5a924947b343a2bdef982012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:32.240021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.240674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.242327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1322/2626] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:31 2022] DELETE /identity/v3/projects/a4b1c18c5a924947b343a2bdef982012 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:32.246281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:32.247615 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:32.258183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9c13-c794-4756-ae42-59f18fb69a19 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:32.311942 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7002493-ada1-4244-9a68-dfcf7dc44754 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:32.317456 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:32.318071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:32.318214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:32.318393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:32.318670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] PATH_INFO: `/v3/users/b541be5ac0e84f369422c546e7ddbf83` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:32.318774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:32.318969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:32.319240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] PATH_INFO: `/v3/users/5803a5d5852d44de817b89ba80e676b4` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:32.319804 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b541be5ac0e84f369422c546e7ddbf83)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:32.320422 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5803a5d5852d44de817b89ba80e676b4)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:32.321732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:32.322471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:32.507342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5803a5d5852d44de817b89ba80e676b4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.508015 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5803a5d5852d44de817b89ba80e676b4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.515307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5803a5d5852d44de817b89ba80e676b4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.527587 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5803a5d5852d44de817b89ba80e676b4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.528264 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5803a5d5852d44de817b89ba80e676b4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.574613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b541be5ac0e84f369422c546e7ddbf83'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.575248 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b541be5ac0e84f369422c546e7ddbf83'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.582023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b541be5ac0e84f369422c546e7ddbf83'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.593460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b541be5ac0e84f369422c546e7ddbf83'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.594171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b541be5ac0e84f369422c546e7ddbf83'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:32.622086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1305/2627] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:32 2022] DELETE /identity/v3/users/5803a5d5852d44de817b89ba80e676b4 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:32.758173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1323/2628] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:32 2022] DELETE /identity/v3/users/b541be5ac0e84f369422c546e7ddbf83 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:33.141697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.143527 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.149021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.150974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.207243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:33.220291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcba54-aaec-4888-ae26-859464cbb980 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:33.226512 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1fb1b-1a27-4ac0-ba95-c308d77a0fd3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:33.231356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:33.231992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.232275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.232554 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] PATH_INFO: `/v3/projects/e71b94d6d9b249eebf347f32935c91d7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.233781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e71b94d6d9b249eebf347f32935c91d7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:33.235710 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:33.245088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:33.284508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:33.284508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.284508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.284508 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.286021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.298174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:33.300122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:33.310128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1306/2629] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:33.310946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.312275 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.324872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabe32f-9e32-45a8-bdb6-83980e3f23aa service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:33.330632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:33.330632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.330632 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.331603 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] PATH_INFO: `/v3/projects/51eece68ed044614a63e101672ea8372` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.332392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51eece68ed044614a63e101672ea8372)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:33.334330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:33.433491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e71b94d6d9b249eebf347f32935c91d7', 'request_id': 'req-0f343bba-384a-4d38-b335-c32f957d1392'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.473226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51eece68ed044614a63e101672ea8372', 'request_id': 'req-5d262ca6-8c4e-420b-ab05-8cd2672afe85'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.518049 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e71b94d6d9b249eebf347f32935c91d7', 'request_id': 'req-0f343bba-384a-4d38-b335-c32f957d1392'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.525892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e71b94d6d9b249eebf347f32935c91d7', 'request_id': 'req-0f343bba-384a-4d38-b335-c32f957d1392'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.537810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] The token cache is being invalidate because project e71b94d6d9b249eebf347f32935c91d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:33.538147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.538878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-0f343bba-384a-4d38-b335-c32f957d1392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.540701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1324/2630] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:33 2022] DELETE /identity/v3/projects/e71b94d6d9b249eebf347f32935c91d7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:33.545203 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6248f-66a4-44a0-adb7-327b825b0329 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.545773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6248f-66a4-44a0-adb7-327b825b0329 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.546126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6248f-66a4-44a0-adb7-327b825b0329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.568180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fbf6248f-66a4-44a0-adb7-327b825b0329 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:33.569408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fbf6248f-66a4-44a0-adb7-327b825b0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.575141 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51eece68ed044614a63e101672ea8372', 'request_id': 'req-5d262ca6-8c4e-420b-ab05-8cd2672afe85'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.582491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51eece68ed044614a63e101672ea8372', 'request_id': 'req-5d262ca6-8c4e-420b-ab05-8cd2672afe85'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.598555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] The token cache is being invalidate because project 51eece68ed044614a63e101672ea8372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:33.598872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.599533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5d262ca6-8c4e-420b-ab05-8cd2672afe85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:33.601067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1307/2631] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:33 2022] DELETE /identity/v3/projects/51eece68ed044614a63e101672ea8372 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:33.605790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa22d7-054f-4fe3-9cba-c964cc903aef None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.606157 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa22d7-054f-4fe3-9cba-c964cc903aef None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.606552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa22d7-054f-4fe3-9cba-c964cc903aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.628122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-acaa22d7-054f-4fe3-9cba-c964cc903aef None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:33.629365 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-acaa22d7-054f-4fe3-9cba-c964cc903aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.645884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1325/2632] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:33.649934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b22e-b7f1-4f32-8227-45cd91bcb830 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.651951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b22e-b7f1-4f32-8227-45cd91bcb830 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.651951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b22e-b7f1-4f32-8227-45cd91bcb830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.655123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1308/2633] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:33.664770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fd409-b14c-4d1c-84a7-7563f24196ee None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.665061 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fd409-b14c-4d1c-84a7-7563f24196ee None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.665402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fd409-b14c-4d1c-84a7-7563f24196ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.673789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-7033b22e-b7f1-4f32-8227-45cd91bcb830 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:33.676490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7033b22e-b7f1-4f32-8227-45cd91bcb830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.684505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1326/2634] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:33.688661 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b22e-b7f1-4f32-8227-45cd91bcb830 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.694983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-ba5fd409-b14c-4d1c-84a7-7563f24196ee None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:33.696234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ba5fd409-b14c-4d1c-84a7-7563f24196ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:33.698767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b22e-b7f1-4f32-8227-45cd91bcb830 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:33.703477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:33.704088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.704368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.704642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.705291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:33.705641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:33.707558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:33.720971 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1327/2635] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:12:33.726842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.737763 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1309/2636] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:33.739316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775786c-ebde-45fc-9764-59ad796080e4 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:33.742049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fd409-b14c-4d1c-84a7-7563f24196ee None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.744292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:33.744985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.745273 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.745551 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.746231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:33.748236 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:33.754132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fd409-b14c-4d1c-84a7-7563f24196ee None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:33.760903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:33.762868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.763577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.763878 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.764933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:33.765074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:33.766636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:33.778758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1310/2637] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:33.784268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.795038 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82819081-356e-459c-8f97-60edc2e60853 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:33.800274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:33.800874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:33.801200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:33.801477 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:33.802185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:33.804125 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:33.932765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1328/2638] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:33.932975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1311/2639] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:33.933065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.933817 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:33.979378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:33.994540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:34.002907 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608a87d-1ed8-438b-9455-53a9c1ff6c58 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.008024 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1a9ef-a5d0-47f5-ab3c-f0767b3e6447 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.008671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.009416 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.009829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.010201 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.011496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.012229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.012832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.013111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.013383 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.013941 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.014061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.015962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.115636 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1312/2640] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:34.116781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.127359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cf66d-9afc-4b9f-b0d1-8c9084584154 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.132257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.132950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.133247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.136984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.136984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.136984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.146018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1329/2641] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:33 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:34.152776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1313/2642] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:34.158224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.159396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.170366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05ad0f-1ebc-4f91-8d25-6cfb3b6fb066 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.170716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089c901-5433-4c3b-86aa-520c75c20b89 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.175559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.176180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.176530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.176701 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.176829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.177502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.177632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.177997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.178366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.179115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.179244 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.180902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.190872 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1330/2643] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:34.192946 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1314/2644] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:34.196818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.197934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.209018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564c742-2bad-4365-97d9-deab29873dd6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.211293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6dc79-9f25-4848-883b-9a81014f8987 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.214596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.215411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ec8da-96e2-4c30-9bde-387b60566a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.215541 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.215897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.216152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ec8da-96e2-4c30-9bde-387b60566a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.216379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ec8da-96e2-4c30-9bde-387b60566a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.216507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.216646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ec8da-96e2-4c30-9bde-387b60566a7e admin admin] PATH_INFO: `/v3/projects/37bf4e32f6d640bebff9a8e1f013c5a7/users/060cac291984412dbecbf42c8f157d28/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.217295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.219154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.232362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1331/2645] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:34.236419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.239258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ec8da-96e2-4c30-9bde-387b60566a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bf4e32f6d640bebff9a8e1f013c5a7, user_id=060cac291984412dbecbf42c8f157d28, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.242066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ec8da-96e2-4c30-9bde-387b60566a7e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.248178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d0597-8d76-4ac2-8ac4-fefe1699af1c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.253073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996eac9-8851-4555-8746-640a7aa7181a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.253889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996eac9-8851-4555-8746-640a7aa7181a admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.254158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996eac9-8851-4555-8746-640a7aa7181a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.254437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996eac9-8851-4555-8746-640a7aa7181a admin admin] PATH_INFO: `/v3/projects/90761a2b55bd4ba18ad11f0938269fbc/users/c27662f0f0fe47ac9445b65599186ac4/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.273249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996eac9-8851-4555-8746-640a7aa7181a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90761a2b55bd4ba18ad11f0938269fbc, user_id=c27662f0f0fe47ac9445b65599186ac4, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.279378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996eac9-8851-4555-8746-640a7aa7181a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.344782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1315/2646] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:34 2022] PUT /identity/v3/projects/37bf4e32f6d640bebff9a8e1f013c5a7/users/060cac291984412dbecbf42c8f157d28/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:34.356361 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc60b6-e363-419e-aab6-801c705de1e5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.356640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc60b6-e363-419e-aab6-801c705de1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.357037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc60b6-e363-419e-aab6-801c705de1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.385569 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1332/2647] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:34 2022] PUT /identity/v3/projects/90761a2b55bd4ba18ad11f0938269fbc/users/c27662f0f0fe47ac9445b65599186ac4/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:34.389161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.389514 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.390048 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.397014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f1cc60b6-e363-419e-aab6-801c705de1e5 None None] MFA Rules not processed for user `060cac291984412dbecbf42c8f157d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:34.398515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f1cc60b6-e363-419e-aab6-801c705de1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:34.418486 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:34.420281 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:34.482498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1316/2648] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:34.486900 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc60b6-e363-419e-aab6-801c705de1e5 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.496084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1333/2649] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:34.506135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.508920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:34.555397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc60b6-e363-419e-aab6-801c705de1e5 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.560877 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.561725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.562099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.562456 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.563151 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.565720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.571380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3e580-b2ed-491d-b296-a331f2fb7ffd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.577817 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.578638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.578638 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.578825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.589094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.590594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.601220 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1334/2650] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:34.685308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1317/2651] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:34 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:34.689304 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.700262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3212f6-be88-4683-985d-4144c82f97d3 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.800953 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.803035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.804973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.804973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.804973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.806097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.903679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1335/2652] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:34 2022] POST /identity/v3/users => generated 299 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:34.907899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.918727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d7a77-beb3-48a2-8290-6f9fd7428692 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.923644 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.925014 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.925696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.925696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.926533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.928069 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:34.939511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1318/2653] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:34.943999 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:34.953922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bdffa-b258-490f-bf1f-add1014a9a21 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:34.960447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:34.961108 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:34.961682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:34.961682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] PATH_INFO: `/v3/projects/31471f9ccb894a29a9ad07b3483c4f3c/users/b6b75ea67a674d8299ec02ae9a5ecb8c/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:34.977321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31471f9ccb894a29a9ad07b3483c4f3c, user_id=b6b75ea67a674d8299ec02ae9a5ecb8c, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:34.980039 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:35.009382 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:35.062826 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:35.069663 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1336/2654] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:34 2022] PUT /identity/v3/projects/31471f9ccb894a29a9ad07b3483c4f3c/users/b6b75ea67a674d8299ec02ae9a5ecb8c/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:35.074413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:35.084638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00888b75-4fb2-4cef-9646-047147688b0d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:35.141815 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a8583-9745-49e1-8570-362d87eac69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:35.143495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8583-9745-49e1-8570-362d87eac69e admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:35.143957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8583-9745-49e1-8570-362d87eac69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:35.144333 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a8583-9745-49e1-8570-362d87eac69e admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:35.145081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a8583-9745-49e1-8570-362d87eac69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:35.146811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a8583-9745-49e1-8570-362d87eac69e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:35.162661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1319/2655] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:35.164652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:35.175018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36656033-c4ea-4571-b028-c068f846dc14 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:35.179933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:35.181011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:35.181011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:35.181351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] PATH_INFO: `/v3/projects/31471f9ccb894a29a9ad07b3483c4f3c/users/b6b75ea67a674d8299ec02ae9a5ecb8c/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:35.183159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31471f9ccb894a29a9ad07b3483c4f3c, user_id=b6b75ea67a674d8299ec02ae9a5ecb8c, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:35.185718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:35.197188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1337/2656] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:35 2022] PUT /identity/v3/projects/31471f9ccb894a29a9ad07b3483c4f3c/users/b6b75ea67a674d8299ec02ae9a5ecb8c/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:35.207599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a44e-5e20-4c82-8069-9a90f9c67705 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:35.209666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a44e-5e20-4c82-8069-9a90f9c67705 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:35.209666 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a44e-5e20-4c82-8069-9a90f9c67705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:35.239053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4a56a44e-5e20-4c82-8069-9a90f9c67705 None None] MFA Rules not processed for user `b6b75ea67a674d8299ec02ae9a5ecb8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:35.240324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4a56a44e-5e20-4c82-8069-9a90f9c67705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:35.342098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1320/2657] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:35.353701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:35.355408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:35.433003 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54c9fd-9acd-4045-a7d5-4e0872efc060 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:35.441307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:35.442218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:35.442504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:35.442822 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:35.528197 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:35.529706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:35.544693 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1338/2658] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:36.359836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc17fca-f154-4336-824d-423c1cf0e93c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:36.360940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc17fca-f154-4336-824d-423c1cf0e93c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:36.361489 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc17fca-f154-4336-824d-423c1cf0e93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:36.399565 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-1cc17fca-f154-4336-824d-423c1cf0e93c None None] MFA Rules not processed for user `c27662f0f0fe47ac9445b65599186ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:36.400825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-1cc17fca-f154-4336-824d-423c1cf0e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:36.487206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:36.497964 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1321/2659] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:36.503112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:36.565265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:36.580777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40960-e532-4af6-8dc6-404a9af60c00 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:36.585459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:36.586109 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:36.586377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:36.586648 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:36.587363 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:36.589249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:36.705040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1339/2660] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:36 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:36.709640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc17fca-f154-4336-824d-423c1cf0e93c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:36.720321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc17fca-f154-4336-824d-423c1cf0e93c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:36.777206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:36.777859 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:36.778137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:36.778407 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:36.779047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:36.780944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:36.937781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1322/2661] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:36 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:36.941684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:36.953604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276376-0c58-42d2-ab08-11be8177d458 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:36.964539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:36.964539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:36.964539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:36.964539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:36.965166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:36.965519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:36.979490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1340/2662] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:36.991128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:37.002631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebac37-ef26-49b1-b24d-af541c89f33a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:37.007298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:37.008042 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:37.008387 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:37.008733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] PATH_INFO: `/v3/projects/12c05dbe3dda47b99feda2f7dd9283e5/users/369a398d1cb04c8993fb442912b49ec5/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:37.027231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c05dbe3dda47b99feda2f7dd9283e5, user_id=369a398d1cb04c8993fb442912b49ec5, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:37.030024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:37.110010 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1323/2663] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:36 2022] PUT /identity/v3/projects/12c05dbe3dda47b99feda2f7dd9283e5/users/369a398d1cb04c8993fb442912b49ec5/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:37.114487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:37.131899 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8807a7a-2b4e-4e97-a2e6-456eaf2cdcff admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:37.191956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:37.192751 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:37.193157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:37.193531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:37.194317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:37.196102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:37.206829 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1341/2664] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:37.211271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:37.221906 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2165c-41f0-4364-8eb2-ada3424ff85c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:37.228196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:37.228811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:37.229084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:37.229356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] PATH_INFO: `/v3/projects/12c05dbe3dda47b99feda2f7dd9283e5/users/369a398d1cb04c8993fb442912b49ec5/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:37.231307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c05dbe3dda47b99feda2f7dd9283e5, user_id=369a398d1cb04c8993fb442912b49ec5, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:37.234150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:37.254177 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1324/2665] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:37 2022] PUT /identity/v3/projects/12c05dbe3dda47b99feda2f7dd9283e5/users/369a398d1cb04c8993fb442912b49ec5/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:12:37.542026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:37.545056 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:37.560047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06775d0-9d9d-4907-ab0f-f47af243a771 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:37.627697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:37.628516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:37.628964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:37.629427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] PATH_INFO: `/v3/users/fe3be97820d64f85aa4d2dbd8f3ffd33` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:37.630659 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3be97820d64f85aa4d2dbd8f3ffd33)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:37.632585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:37.819380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3be97820d64f85aa4d2dbd8f3ffd33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:37.820032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3be97820d64f85aa4d2dbd8f3ffd33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:37.830718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3be97820d64f85aa4d2dbd8f3ffd33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:37.833260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:37.848546 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3be97820d64f85aa4d2dbd8f3ffd33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:37.849606 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-c56f2bed-e792-41a4-85eb-476f7b45a56e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3be97820d64f85aa4d2dbd8f3ffd33'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:37.940624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1342/2666] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:37 2022] DELETE /identity/v3/users/fe3be97820d64f85aa4d2dbd8f3ffd33 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:38.322870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:38.324976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:38.341562 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d89a91-1a7c-455c-bd35-22168d20c902 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:38.417034 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:38.417679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:38.418011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:38.418439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] PATH_INFO: `/v3/projects/a4e94c8248874defa3cb01d0725dfcb0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:38.419528 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e94c8248874defa3cb01d0725dfcb0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:38.421497 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:38.521902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e94c8248874defa3cb01d0725dfcb0', 'request_id': 'req-68b4c362-4505-4b58-aaa3-df8d19f312c9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:38.524057 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:38.629413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e94c8248874defa3cb01d0725dfcb0', 'request_id': 'req-68b4c362-4505-4b58-aaa3-df8d19f312c9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:38.637989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e94c8248874defa3cb01d0725dfcb0', 'request_id': 'req-68b4c362-4505-4b58-aaa3-df8d19f312c9'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:38.654903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] The token cache is being invalidate because project a4e94c8248874defa3cb01d0725dfcb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:38.655358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:38.656090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-68b4c362-4505-4b58-aaa3-df8d19f312c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:38.658024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1325/2667] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:38 2022] DELETE /identity/v3/projects/a4e94c8248874defa3cb01d0725dfcb0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:38.663097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9a9d0-22e7-4fa8-b232-d268aec6fa6c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:38.663613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9a9d0-22e7-4fa8-b232-d268aec6fa6c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:38.667592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9a9d0-22e7-4fa8-b232-d268aec6fa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:38.684436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-38c9a9d0-22e7-4fa8-b232-d268aec6fa6c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:38.685688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-38c9a9d0-22e7-4fa8-b232-d268aec6fa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:38.710515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:38.710955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:38.711322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:38.743429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] MFA Rules not processed for user `369a398d1cb04c8993fb442912b49ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:38.744721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:38.768192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1343/2668] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:38.781027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef9a32-cbc3-4481-a6ea-7639e2395d7d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:38.781335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef9a32-cbc3-4481-a6ea-7639e2395d7d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:38.781903 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef9a32-cbc3-4481-a6ea-7639e2395d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:38.804906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-61ef9a32-cbc3-4481-a6ea-7639e2395d7d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:38.806316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-61ef9a32-cbc3-4481-a6ea-7639e2395d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:38.814660 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1344/2669] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:38.818978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef9a32-cbc3-4481-a6ea-7639e2395d7d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:38.822751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1326/2670] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:38.828295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef9a32-cbc3-4481-a6ea-7639e2395d7d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:38.834012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:38.834612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:38.834889 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:38.835158 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:38.835798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:38.836119 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:38.837675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:38.839573 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:38.841110 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:38.860662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1345/2671] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:12:38.860662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:38.865134 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075d42a-ea72-4fac-9e13-039313e45512 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:38.869613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:38.870226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:38.870500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:38.870770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:38.871389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:38.873264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:38.901352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c36b9-86ed-4f5e-87f2-3b051a6d0b5c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:38.905035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:38.905645 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:38.905956 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:38.906231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:38.908242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:38.966678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:38.968109 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:38.976767 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1327/2672] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:38.990429 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1346/2673] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:38 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:38.994811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.005851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecfaba-0fc3-49ae-910b-8b226bfd498e service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.061099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.061824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.062197 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.062589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.063335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.066024 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:39.166432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1328/2674] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:38 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:39.170654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.181096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577597fa-3595-4d71-ae8d-4c5a26780b2c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.185836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.186427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.186701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.186973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.187605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.189184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:39.200015 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1347/2675] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:39.207940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.217920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7ad3e-6d9b-4ad9-91db-db33987be28d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.222574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a3ad0-a58c-4746-848a-00ac0607e72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.223168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3ad0-a58c-4746-848a-00ac0607e72f admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.223449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3ad0-a58c-4746-848a-00ac0607e72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.223723 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3ad0-a58c-4746-848a-00ac0607e72f admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.224362 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a3ad0-a58c-4746-848a-00ac0607e72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.226035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a3ad0-a58c-4746-848a-00ac0607e72f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:39.235741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1329/2676] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:39.241591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.251534 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbc01c-289d-4905-903d-9e4addb13b9e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.256772 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.257416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.257683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.257983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] PATH_INFO: `/v3/projects/22c6a340b9b94a329f62f085859f4668/users/cc8dcc2f2793432aa2c21f7e47600258/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.272915 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] RBAC: Authorizing `identity:create_grant(project_id=22c6a340b9b94a329f62f085859f4668, user_id=cc8dcc2f2793432aa2c21f7e47600258, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.275549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:39.349843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1348/2677] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:39 2022] PUT /identity/v3/projects/22c6a340b9b94a329f62f085859f4668/users/cc8dcc2f2793432aa2c21f7e47600258/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:39.362294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078bfba-eb21-4afe-8007-19df196c9e0b None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.362796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078bfba-eb21-4afe-8007-19df196c9e0b None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.362915 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078bfba-eb21-4afe-8007-19df196c9e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.391286 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7078bfba-eb21-4afe-8007-19df196c9e0b None None] MFA Rules not processed for user `cc8dcc2f2793432aa2c21f7e47600258`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:39.392525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7078bfba-eb21-4afe-8007-19df196c9e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:39.457520 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1330/2678] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:39.462268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.511839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:39.530424 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d28b8-1c5a-401e-8edd-7480bfcd403f admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.535471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.536066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.536342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.536612 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.537260 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.539148 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:39.543482 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:39.655995 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1349/2679] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:39 2022] POST /identity/v3/projects => generated 365 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:39.678919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078bfba-eb21-4afe-8007-19df196c9e0b None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.685008 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078bfba-eb21-4afe-8007-19df196c9e0b None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.752983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.753708 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.754101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.754440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.755248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.757316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:39.877613 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1331/2680] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:39 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:39.881784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.892073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c5cbc-3f06-4141-8382-5f6ec544f8cb admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.896818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.897411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.897687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.897986 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.898624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.900205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:39.911641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1350/2681] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:39.915994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:39.926583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2506e8-c98b-482b-b784-a6158ad756a2 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:39.931183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:39.931772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:39.932052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:39.932328 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] PATH_INFO: `/v3/projects/425654dc63dc46b0860e3c86f7c0a75b/users/45c4e4ceeab24bb48507e74e9f739701/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:39.949530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425654dc63dc46b0860e3c86f7c0a75b, user_id=45c4e4ceeab24bb48507e74e9f739701, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:39.952267 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:40.054799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1332/2682] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:39 2022] PUT /identity/v3/projects/425654dc63dc46b0860e3c86f7c0a75b/users/45c4e4ceeab24bb48507e74e9f739701/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:40.059029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:40.075263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d641f73-6778-49bb-8104-eded8186a5a2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:40.128861 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0729c-0dcf-43df-9b82-334b7c69077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:40.129454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0729c-0dcf-43df-9b82-334b7c69077c admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:40.129761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0729c-0dcf-43df-9b82-334b7c69077c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:40.130035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0729c-0dcf-43df-9b82-334b7c69077c admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:40.130706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0729c-0dcf-43df-9b82-334b7c69077c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:40.132283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0729c-0dcf-43df-9b82-334b7c69077c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:40.142081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1351/2683] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:40.146864 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:40.159379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0aaa5-a2eb-4e64-bf49-4bd2682286d9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:40.164019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:40.166373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:40.167229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:40.167515 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:40.167794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] PATH_INFO: `/v3/projects/425654dc63dc46b0860e3c86f7c0a75b/users/45c4e4ceeab24bb48507e74e9f739701/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:40.169724 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425654dc63dc46b0860e3c86f7c0a75b, user_id=45c4e4ceeab24bb48507e74e9f739701, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:40.172460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:40.183532 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1333/2684] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:40 2022] PUT /identity/v3/projects/425654dc63dc46b0860e3c86f7c0a75b/users/45c4e4ceeab24bb48507e74e9f739701/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:40.193860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd44e0-d0c3-4339-a1b9-321501d4d7cc None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:40.194170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd44e0-d0c3-4339-a1b9-321501d4d7cc None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:40.194511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd44e0-d0c3-4339-a1b9-321501d4d7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:40.222742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-dbcd44e0-d0c3-4339-a1b9-321501d4d7cc None None] MFA Rules not processed for user `45c4e4ceeab24bb48507e74e9f739701`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:40.223998 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dbcd44e0-d0c3-4339-a1b9-321501d4d7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:40.302851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1352/2685] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:40.314406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:40.316316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:40.373930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e79d0-16de-48bb-9a9a-baa9414a76e9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:40.378292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:40.378909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:40.379186 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:40.379458 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:40.388424 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:40.389947 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:40.400163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1334/2686] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:40.736706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd44e0-d0c3-4339-a1b9-321501d4d7cc None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:40.747038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd44e0-d0c3-4339-a1b9-321501d4d7cc None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:40.750649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533a4b-391f-4c9b-9eee-079763a72f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:40.752575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533a4b-391f-4c9b-9eee-079763a72f90 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:40.752575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533a4b-391f-4c9b-9eee-079763a72f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:40.752575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533a4b-391f-4c9b-9eee-079763a72f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:40.804379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:40.815293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533a4b-391f-4c9b-9eee-079763a72f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:40.816779 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533a4b-391f-4c9b-9eee-079763a72f90 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:40.826312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1353/2687] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:41.451665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:41.513124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517140-8750-4b0e-829b-664485efa7a9 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:41.517551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:41.518348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:41.518626 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:41.519369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] PATH_INFO: `/v3/users/cc8dcc2f2793432aa2c21f7e47600258` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:41.520474 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8dcc2f2793432aa2c21f7e47600258)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:41.522411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:41.742697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8dcc2f2793432aa2c21f7e47600258'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:41.743351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8dcc2f2793432aa2c21f7e47600258'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:41.750394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8dcc2f2793432aa2c21f7e47600258'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:41.762302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8dcc2f2793432aa2c21f7e47600258'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:41.762945 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8dcc2f2793432aa2c21f7e47600258'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:41.838176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:41.842405 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1335/2688] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:41 2022] DELETE /identity/v3/users/cc8dcc2f2793432aa2c21f7e47600258 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:41.847679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:41.848136 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:41.848231 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:41.867344 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:41.868539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:41.927540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1354/2689] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:41.937278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:41.940095 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:41.995739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d29709-543d-45bd-b12b-b4b0b20d1e42 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:41.999359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:41.999957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:42.000234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:42.000558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:42.010263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:42.011714 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:42.022380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1336/2690] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:42.513390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:42.514980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:42.527423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa05a5-ea05-47ef-b501-a96dfe756318 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:42.532493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:42.533125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:42.533718 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:42.534080 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] PATH_INFO: `/v3/projects/22c6a340b9b94a329f62f085859f4668` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:42.535191 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] RBAC: Authorizing `identity:delete_project(project_id=22c6a340b9b94a329f62f085859f4668)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:42.537079 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:42.649798 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22c6a340b9b94a329f62f085859f4668', 'request_id': 'req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:42.722565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22c6a340b9b94a329f62f085859f4668', 'request_id': 'req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:42.729190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22c6a340b9b94a329f62f085859f4668', 'request_id': 'req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:42.740073 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] The token cache is being invalidate because project 22c6a340b9b94a329f62f085859f4668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:42.740362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:42.740991 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:42.742570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1355/2691] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:42 2022] DELETE /identity/v3/projects/22c6a340b9b94a329f62f085859f4668 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:42.746592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:42.756501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c47e3b-2a2b-40ed-aab5-95d2a32ca98e service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:42.812806 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:42.813404 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:42.813686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:42.813994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] PATH_INFO: `/v3/users/45c4e4ceeab24bb48507e74e9f739701` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:42.815411 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c4e4ceeab24bb48507e74e9f739701)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:42.817377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:42.993899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c4e4ceeab24bb48507e74e9f739701'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:42.994546 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c4e4ceeab24bb48507e74e9f739701'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.001570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c4e4ceeab24bb48507e74e9f739701'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.013787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c4e4ceeab24bb48507e74e9f739701'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.014425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-5fda6f91-1215-497e-bd58-f77a153bd243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c4e4ceeab24bb48507e74e9f739701'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.088369 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1337/2692] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:42 2022] DELETE /identity/v3/users/45c4e4ceeab24bb48507e74e9f739701 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:43.568150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:43.569721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:43.595298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37de5e-5ecf-41b0-9dfb-545b5d31068e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:43.662662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:43.663372 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:43.663650 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:43.663927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] PATH_INFO: `/v3/projects/425654dc63dc46b0860e3c86f7c0a75b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:43.665095 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=425654dc63dc46b0860e3c86f7c0a75b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:43.667176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:43.682657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:43.807965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425654dc63dc46b0860e3c86f7c0a75b', 'request_id': 'req-e250b128-5af4-4ec4-94b1-374e69174260'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.926083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425654dc63dc46b0860e3c86f7c0a75b', 'request_id': 'req-e250b128-5af4-4ec4-94b1-374e69174260'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.933350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425654dc63dc46b0860e3c86f7c0a75b', 'request_id': 'req-e250b128-5af4-4ec4-94b1-374e69174260'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.944896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] The token cache is being invalidate because project 425654dc63dc46b0860e3c86f7c0a75b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:43.945184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.945907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-e250b128-5af4-4ec4-94b1-374e69174260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:43.947384 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1356/2693] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:43 2022] DELETE /identity/v3/projects/425654dc63dc46b0860e3c86f7c0a75b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:43.952377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d84a0-8ab5-418c-87b5-e4fdf8fd4b40 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:43.952841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d84a0-8ab5-418c-87b5-e4fdf8fd4b40 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:43.952974 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d84a0-8ab5-418c-87b5-e4fdf8fd4b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:43.972687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-823d84a0-8ab5-418c-87b5-e4fdf8fd4b40 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:43.973926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-823d84a0-8ab5-418c-87b5-e4fdf8fd4b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:43.990748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:44.031437 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1338/2694] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:44.039686 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c9081-0e8a-42e6-9423-82274df3d3d4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.039968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c9081-0e8a-42e6-9423-82274df3d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.040240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c9081-0e8a-42e6-9423-82274df3d3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.061330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-a56c9081-0e8a-42e6-9423-82274df3d3d4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:44.062580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a56c9081-0e8a-42e6-9423-82274df3d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:44.070886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1357/2695] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:44.075101 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d84a0-8ab5-418c-87b5-e4fdf8fd4b40 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:44.085174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d84a0-8ab5-418c-87b5-e4fdf8fd4b40 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:44.089732 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:44.090359 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.090638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.090931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.091579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:44.091884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:44.093509 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:44.105571 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1339/2696] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:44.115581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c9081-0e8a-42e6-9423-82274df3d3d4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:44.119176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c9081-0e8a-42e6-9423-82274df3d3d4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:44.123761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:44.124350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.124624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.124892 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.125511 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:44.127519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:44.275101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1358/2697] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:44 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:44.276376 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:44.287212 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814544-ce9f-41a5-b543-c93664585a5c admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:44.347428 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:44.348041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.348302 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.348574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.349237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:44.351250 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:44.467373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1340/2698] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:44 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:44.476530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:44.489360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611f6ee-b02c-4bc8-a37b-d636e99298e7 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:44.495101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:44.495778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.496121 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.496553 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.497316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:44.499127 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:44.509491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1359/2699] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:44.513712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:44.523805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dcb85-c37f-4534-a9bc-9072251b5d5a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:44.528400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079712-3b67-43a5-a2b0-8916ac83cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:44.528991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079712-3b67-43a5-a2b0-8916ac83cfdc admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.529268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079712-3b67-43a5-a2b0-8916ac83cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.529537 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079712-3b67-43a5-a2b0-8916ac83cfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.530268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079712-3b67-43a5-a2b0-8916ac83cfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:44.531882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079712-3b67-43a5-a2b0-8916ac83cfdc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:44.543288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1341/2700] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:12:44.547519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:44.557983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfd540-b97e-4f02-a257-96b39a898431 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:44.562671 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:44.563317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.563649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.563926 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] PATH_INFO: `/v3/projects/ad9294db70064b40b9b16468786a2010/users/faa041e2b0e3450b88234011945183d7/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.578681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9294db70064b40b9b16468786a2010, user_id=faa041e2b0e3450b88234011945183d7, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:44.581935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:44.686989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1360/2701] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:44 2022] PUT /identity/v3/projects/ad9294db70064b40b9b16468786a2010/users/faa041e2b0e3450b88234011945183d7/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:44.703430 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723419-ab73-4a6c-a3fb-0c2317a66ad4 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.704912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723419-ab73-4a6c-a3fb-0c2317a66ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.704912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723419-ab73-4a6c-a3fb-0c2317a66ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.734272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5e723419-ab73-4a6c-a3fb-0c2317a66ad4 None None] MFA Rules not processed for user `faa041e2b0e3450b88234011945183d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:44.735469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5e723419-ab73-4a6c-a3fb-0c2317a66ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:44.803124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1342/2702] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:44.807847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:44.871706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce4194-bf06-4a3f-a6a9-abddfaf82c24 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:44.876849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:44.877757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:44.878040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:44.878314 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:44.879020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:44.880936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:44.999802 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1361/2703] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:44 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:45.002062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723419-ab73-4a6c-a3fb-0c2317a66ad4 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.013273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723419-ab73-4a6c-a3fb-0c2317a66ad4 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.026240 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:45.071731 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.072367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.072648 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.072919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.073548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.075548 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.228011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1343/2704] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:44 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:45.232269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.244911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d73dd-1825-4c67-9b56-a54d0273db8c admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.250611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.251355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.251767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.252183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.253070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.254675 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.265557 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1362/2705] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:45.269873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.280155 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19c9c4-0d36-421c-a969-a34d9c10d5d6 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.285820 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.286488 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.286825 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.287027 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.289400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.289400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.300202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1344/2706] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:45.304430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.314763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86600863-048f-479e-bde3-e8033146fc7e admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.319350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8cbf5-4478-4801-a79c-134b4750663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.320000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8cbf5-4478-4801-a79c-134b4750663e admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.320287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8cbf5-4478-4801-a79c-134b4750663e admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.320556 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8cbf5-4478-4801-a79c-134b4750663e admin admin] PATH_INFO: `/v3/projects/55e6d85478f9486cbe7a5a9ab6080c9c/users/d074109a14a94ab583aa1de65f8e734b/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.332581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:45.336933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8cbf5-4478-4801-a79c-134b4750663e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e6d85478f9486cbe7a5a9ab6080c9c, user_id=d074109a14a94ab583aa1de65f8e734b, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.339555 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8cbf5-4478-4801-a79c-134b4750663e admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.366919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.369051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:45.427491 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef3963-00c0-40bf-9079-1e642e67bbb8 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.431697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1363/2707] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:45 2022] PUT /identity/v3/projects/55e6d85478f9486cbe7a5a9ab6080c9c/users/d074109a14a94ab583aa1de65f8e734b/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:45.442451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd97d-10d1-431e-bff0-5fb7b1aae81c None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.442736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd97d-10d1-431e-bff0-5fb7b1aae81c None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.443009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd97d-10d1-431e-bff0-5fb7b1aae81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.470420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-163cd97d-10d1-431e-bff0-5fb7b1aae81c None None] MFA Rules not processed for user `d074109a14a94ab583aa1de65f8e734b`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:45.472373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-163cd97d-10d1-431e-bff0-5fb7b1aae81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:45.486786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.487494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.487773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.488041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.490183 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:45.541272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1364/2708] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:45.546292 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cd97d-10d1-431e-bff0-5fb7b1aae81c None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.557646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.559116 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.568982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1345/2709] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:12:45.608887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cd97d-10d1-431e-bff0-5fb7b1aae81c None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.613684 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.614321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.614604 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.614878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.615633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.617646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.725163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1365/2710] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:45 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:12:45.729318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.734764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:45.740870 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315095e-3371-44be-8f2c-16b85405041e service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.798225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.798855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.799130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.799579 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.800215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.802231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.898351 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1346/2711] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:45 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:45.902491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.913257 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42d9de-40df-4b08-93c2-f97951936404 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.917936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.918558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.918868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.919138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.919771 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.921352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:45.931761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1366/2712] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:45.935981 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:45.947132 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d539748-d75e-43d4-aa8a-f7745d257b29 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:45.951924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:45.956323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:45.956323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:45.956323 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] PATH_INFO: `/v3/projects/07cc4c6bf07f4f479e6ec525186fcd9e/users/02f52107d00746e08b98de2331fdee08/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:45.971680 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cc4c6bf07f4f479e6ec525186fcd9e, user_id=02f52107d00746e08b98de2331fdee08, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:45.974389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:46.066181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1347/2713] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:45 2022] PUT /identity/v3/projects/07cc4c6bf07f4f479e6ec525186fcd9e/users/02f52107d00746e08b98de2331fdee08/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:46.070183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:46.080221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c374f91-9a19-48a3-bc63-7e078ed8f4d3 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:46.128831 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:46.138970 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77353524-0cec-438a-a495-76e175e4e181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:46.139626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77353524-0cec-438a-a495-76e175e4e181 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:46.139902 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77353524-0cec-438a-a495-76e175e4e181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:46.140178 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77353524-0cec-438a-a495-76e175e4e181 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:46.140823 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77353524-0cec-438a-a495-76e175e4e181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:46.142469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77353524-0cec-438a-a495-76e175e4e181 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:46.154086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1367/2714] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:46.161471 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:46.172441 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfee859-2806-4e66-82f7-cde1175f90f7 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:46.178352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:46.179002 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:46.179343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:46.179657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] PATH_INFO: `/v3/projects/07cc4c6bf07f4f479e6ec525186fcd9e/users/02f52107d00746e08b98de2331fdee08/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:46.181581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cc4c6bf07f4f479e6ec525186fcd9e, user_id=02f52107d00746e08b98de2331fdee08, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:46.184225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:46.196368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1348/2715] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:46 2022] PUT /identity/v3/projects/07cc4c6bf07f4f479e6ec525186fcd9e/users/02f52107d00746e08b98de2331fdee08/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:46.206963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:46.207249 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:46.207530 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:46.235470 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] MFA Rules not processed for user `02f52107d00746e08b98de2331fdee08`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:46.237403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:46.310577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1368/2716] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:46.322616 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:46.378049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65945a71-faa3-4cb6-a2ba-226ba6c012e4 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:46.382078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:46.382675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:46.382950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:46.383219 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:46.392322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:46.393835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:46.403049 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1349/2717] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:47.405222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:47.406657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:47.419038 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43934a-b11e-4eac-97cb-b87749a2f4c6 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:47.422894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:47.423490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:47.423766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:47.424035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:47.481135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:47.482601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:47.492921 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1369/2718] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:52.596278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:52.606473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db5cf2-91ff-465f-9a49-de2cfd6fc4f5 service cinder] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:52.610670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fc46b-1fb3-4b5f-b664-f749537b7ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:52.612096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fc46b-1fb3-4b5f-b664-f749537b7ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:52.612096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fc46b-1fb3-4b5f-b664-f749537b7ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:52.612096 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fc46b-1fb3-4b5f-b664-f749537b7ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:52.688990 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:52.692995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fc46b-1fb3-4b5f-b664-f749537b7ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:52.694592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fc46b-1fb3-4b5f-b664-f749537b7ea5 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:52.706619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1350/2719] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:52.931896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:52.933419 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:52.999154 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bb281-c779-482c-bae2-606e3af3a8b3 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:53.004490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:53.008985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:53.008985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:53.008985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] PATH_INFO: `/v3/users/060cac291984412dbecbf42c8f157d28` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:53.008985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060cac291984412dbecbf42c8f157d28)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:53.008985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:53.023298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:53.224739 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060cac291984412dbecbf42c8f157d28'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:53.225399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060cac291984412dbecbf42c8f157d28'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:53.232859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060cac291984412dbecbf42c8f157d28'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:53.245931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060cac291984412dbecbf42c8f157d28'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:53.246640 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060cac291984412dbecbf42c8f157d28'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:53.339523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1370/2720] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:52 2022] DELETE /identity/v3/users/060cac291984412dbecbf42c8f157d28 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:53.345470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:53.345868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:53.346230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:53.366193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:53.367421 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:53.427170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1351/2721] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:53.436004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:53.437480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:53.505210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc33646-16b0-4093-be06-93f4d04a4a38 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:53.505210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:53.505210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:53.505210 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:53.506453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:53.512297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:53.513832 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:53.524156 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1371/2722] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:53.980638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:53.982142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:53.994498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a967700-1158-41b2-8b9d-db6fdcb245f0 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:53.999412 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:54.000102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:54.000469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:54.005995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] PATH_INFO: `/v3/projects/37bf4e32f6d640bebff9a8e1f013c5a7` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:54.005995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bf4e32f6d640bebff9a8e1f013c5a7)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:54.005995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:54.116564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bf4e32f6d640bebff9a8e1f013c5a7', 'request_id': 'req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.236960 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bf4e32f6d640bebff9a8e1f013c5a7', 'request_id': 'req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.245260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bf4e32f6d640bebff9a8e1f013c5a7', 'request_id': 'req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.259350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] The token cache is being invalidate because project 37bf4e32f6d640bebff9a8e1f013c5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:54.259939 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.260828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.262812 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1352/2723] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:53 2022] DELETE /identity/v3/projects/37bf4e32f6d640bebff9a8e1f013c5a7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:54.266875 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:54.281373 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e087d-a74d-4c04-9ef4-e1b5974931fa service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:54.370133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:54.370778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:54.371060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:54.371335 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] PATH_INFO: `/v3/users/b6b75ea67a674d8299ec02ae9a5ecb8c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:54.372481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b75ea67a674d8299ec02ae9a5ecb8c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:54.374518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:54.685426 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:54.695221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b75ea67a674d8299ec02ae9a5ecb8c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.695907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b75ea67a674d8299ec02ae9a5ecb8c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.703057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b75ea67a674d8299ec02ae9a5ecb8c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.716885 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b75ea67a674d8299ec02ae9a5ecb8c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.717605 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5cea0db0-8561-4cde-befe-7d89ddec5749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b75ea67a674d8299ec02ae9a5ecb8c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:54.800816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1372/2724] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:54 2022] DELETE /identity/v3/users/b6b75ea67a674d8299ec02ae9a5ecb8c => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:55.232544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:55.235022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:55.315510 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9bfb-bf98-4b19-acd4-1c15ee45f37f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:55.328330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:55.372260 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:55.372794 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:55.373102 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:55.373379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] PATH_INFO: `/v3/projects/31471f9ccb894a29a9ad07b3483c4f3c` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:55.374551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=31471f9ccb894a29a9ad07b3483c4f3c)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:55.376457 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:55.522200 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31471f9ccb894a29a9ad07b3483c4f3c', 'request_id': 'req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:55.613913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31471f9ccb894a29a9ad07b3483c4f3c', 'request_id': 'req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:55.620679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31471f9ccb894a29a9ad07b3483c4f3c', 'request_id': 'req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:55.631600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] The token cache is being invalidate because project 31471f9ccb894a29a9ad07b3483c4f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:55.631811 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:55.632473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-e7361ccd-53c8-4d13-90fd-b0851a8ae3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:55.633991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1353/2725] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:55 2022] DELETE /identity/v3/projects/31471f9ccb894a29a9ad07b3483c4f3c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:55.639783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8ddc-b54e-4998-a93d-85a67cd07c55 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:55.640205 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8ddc-b54e-4998-a93d-85a67cd07c55 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:55.640525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8ddc-b54e-4998-a93d-85a67cd07c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:55.700790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fced8ddc-b54e-4998-a93d-85a67cd07c55 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:55.702043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fced8ddc-b54e-4998-a93d-85a67cd07c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:55.774208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1373/2726] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:55.835074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694889-cd25-493f-96e3-0ed49a8407f5 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:55.835364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694889-cd25-493f-96e3-0ed49a8407f5 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:55.835639 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694889-cd25-493f-96e3-0ed49a8407f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:55.856303 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-a1694889-cd25-493f-96e3-0ed49a8407f5 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:55.857572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a1694889-cd25-493f-96e3-0ed49a8407f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:55.865716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1354/2727] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:55.870276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8ddc-b54e-4998-a93d-85a67cd07c55 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:55.880181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8ddc-b54e-4998-a93d-85a67cd07c55 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:55.885128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:55.885790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:55.886066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:55.886342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:55.887032 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:12:55.887342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:55.888968 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:55.901694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1374/2728] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:12:55.906214 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1694889-cd25-493f-96e3-0ed49a8407f5 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:55.916436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1694889-cd25-493f-96e3-0ed49a8407f5 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:55.921272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:55.921901 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:55.924866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:55.924866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:55.924866 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:55.925135 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:56.042709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1355/2729] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:12:55 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:56.043843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:56.054914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4b9c-71c9-428f-894b-34dc919dce5d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:56.115150 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:56.115757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:56.116037 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:56.116308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:56.116932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:56.118894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:56.203933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:56.212268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1375/2730] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:12:56 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:12:56.214088 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:56.226716 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7ea28-c64e-4a35-b438-4f54004a0af3 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:56.231938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:56.232621 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:56.243432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:56.243432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:56.243432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:56.243432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:56.247557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1356/2731] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:56.251696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:56.262679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24e8ab-b973-4084-a6a3-61eef3438959 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:56.267347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caafc85-bbe7-4aff-8cf4-8898fffb75fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:56.268061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caafc85-bbe7-4aff-8cf4-8898fffb75fa admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:56.268381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caafc85-bbe7-4aff-8cf4-8898fffb75fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:56.268616 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caafc85-bbe7-4aff-8cf4-8898fffb75fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:56.269256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caafc85-bbe7-4aff-8cf4-8898fffb75fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:56.271576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caafc85-bbe7-4aff-8cf4-8898fffb75fa admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:56.282430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1376/2732] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:12:56.286994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:56.296846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349be6-f097-4e1c-8ebb-2880444d4ee1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:56.301578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa598b88-df85-42db-9838-30067342822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:56.302247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa598b88-df85-42db-9838-30067342822f admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:56.302530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa598b88-df85-42db-9838-30067342822f admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:56.302803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa598b88-df85-42db-9838-30067342822f admin admin] PATH_INFO: `/v3/projects/d63cdb20ba4b46af98b88cb9867fd275/users/bea0e9183c47439f81544c5eb9cf6c77/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:56.320653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa598b88-df85-42db-9838-30067342822f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63cdb20ba4b46af98b88cb9867fd275, user_id=bea0e9183c47439f81544c5eb9cf6c77, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:56.323448 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa598b88-df85-42db-9838-30067342822f admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:56.426517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1357/2733] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:12:56 2022] PUT /identity/v3/projects/d63cdb20ba4b46af98b88cb9867fd275/users/bea0e9183c47439f81544c5eb9cf6c77/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:56.437490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:56.437770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:56.438067 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:56.488580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] MFA Rules not processed for user `bea0e9183c47439f81544c5eb9cf6c77`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:56.490065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:56.560430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1377/2734] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:12:56.574410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa598b88-df85-42db-9838-30067342822f admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:56.576321 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-fa598b88-df85-42db-9838-30067342822f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:56.639614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:56.664280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa598b88-df85-42db-9838-30067342822f admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:56.668190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d2be4-d85d-453f-8dce-6211410f48e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:56.668734 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d2be4-d85d-453f-8dce-6211410f48e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:56.669039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d2be4-d85d-453f-8dce-6211410f48e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:56.669308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d2be4-d85d-453f-8dce-6211410f48e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:56.678343 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d2be4-d85d-453f-8dce-6211410f48e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:56.679919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d2be4-d85d-453f-8dce-6211410f48e9 service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:56.688938 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1358/2735] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:57.279575 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:57.280962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:57.351066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369e708-37d7-4d38-8193-1f29ef4a73aa None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:57.355735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:57.356402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:57.356674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:57.356952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] PATH_INFO: `/v3/users/faa041e2b0e3450b88234011945183d7` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:57.358102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faa041e2b0e3450b88234011945183d7)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:57.360071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:57.626871 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faa041e2b0e3450b88234011945183d7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:57.627538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faa041e2b0e3450b88234011945183d7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:57.642347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faa041e2b0e3450b88234011945183d7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:57.657222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faa041e2b0e3450b88234011945183d7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:57.658012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faa041e2b0e3450b88234011945183d7'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:57.724445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:57.732674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1378/2736] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:57 2022] DELETE /identity/v3/users/faa041e2b0e3450b88234011945183d7 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:57.737068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:57.737596 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:57.737720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:57.758048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:12:57.759335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:57.817829 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1359/2737] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:12:57.827471 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:57.828961 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:57.893128 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402acc0-279b-4cbc-8b6c-8f4751523927 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:57.896825 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:57.897437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:57.897719 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:57.898030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:57.907088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:57.910522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:57.918317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1379/2738] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:12:58.415918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:58.421867 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:58.430453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b65f63-b866-4fb3-ac58-456bd6fba040 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:58.435236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:58.435896 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:58.436168 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:58.436436 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] PATH_INFO: `/v3/projects/ad9294db70064b40b9b16468786a2010` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:58.437916 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9294db70064b40b9b16468786a2010)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:58.439913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:58.595963 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9294db70064b40b9b16468786a2010', 'request_id': 'req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:58.676674 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9294db70064b40b9b16468786a2010', 'request_id': 'req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:58.684117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9294db70064b40b9b16468786a2010', 'request_id': 'req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:58.696357 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] The token cache is being invalidate because project ad9294db70064b40b9b16468786a2010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:12:58.696653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:58.697292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:58.698937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1360/2739] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:58 2022] DELETE /identity/v3/projects/ad9294db70064b40b9b16468786a2010 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:58.702781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:58.714993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e537dec-7e63-487c-ba5b-64b66f641856 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:58.792141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:58.792777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:58.793063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:58.793367 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] PATH_INFO: `/v3/users/d074109a14a94ab583aa1de65f8e734b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:58.794561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d074109a14a94ab583aa1de65f8e734b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:58.796591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:12:59.256250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd074109a14a94ab583aa1de65f8e734b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:59.256893 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd074109a14a94ab583aa1de65f8e734b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:59.263870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd074109a14a94ab583aa1de65f8e734b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:59.276632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd074109a14a94ab583aa1de65f8e734b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:59.277277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd074109a14a94ab583aa1de65f8e734b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:12:59.398242 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1380/2740] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:12:58 2022] DELETE /identity/v3/users/d074109a14a94ab583aa1de65f8e734b => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:12:59.844397 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:12:59.846338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:12:59.862583 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8ce44-cabf-40c0-923f-2dd08d2c0f20 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:12:59.927913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:12:59.943500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:12:59.946346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:12:59.946346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:12:59.946346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] PATH_INFO: `/v3/projects/55e6d85478f9486cbe7a5a9ab6080c9c` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:12:59.948070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e6d85478f9486cbe7a5a9ab6080c9c)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:12:59.950355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:00.060265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e6d85478f9486cbe7a5a9ab6080c9c', 'request_id': 'req-cb001412-5c03-4429-8a24-9ff98975b0a8'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.148758 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e6d85478f9486cbe7a5a9ab6080c9c', 'request_id': 'req-cb001412-5c03-4429-8a24-9ff98975b0a8'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.155857 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e6d85478f9486cbe7a5a9ab6080c9c', 'request_id': 'req-cb001412-5c03-4429-8a24-9ff98975b0a8'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.166751 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] The token cache is being invalidate because project 55e6d85478f9486cbe7a5a9ab6080c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:00.167036 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.167679 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.169158 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1361/2741] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:12:59 2022] DELETE /identity/v3/projects/55e6d85478f9486cbe7a5a9ab6080c9c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:00.173253 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:00.174732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:00.187387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9526f-bf71-461f-ac88-62c7b701508a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:00.188683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:00.190060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:00.205165 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:00.247941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:00.249924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:00.249924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:00.249924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] PATH_INFO: `/v3/users/02f52107d00746e08b98de2331fdee08` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:00.250316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f52107d00746e08b98de2331fdee08)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:00.250937 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb001412-5c03-4429-8a24-9ff98975b0a8 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:00.252271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:00.256327 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:00.257249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:00.258090 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:00.258540 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:00.261048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:00.326266 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:00.327756 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:00.336851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1362/2742] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:00.503183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f52107d00746e08b98de2331fdee08'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.504732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f52107d00746e08b98de2331fdee08'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.511737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f52107d00746e08b98de2331fdee08'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.523637 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f52107d00746e08b98de2331fdee08'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.524350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f52107d00746e08b98de2331fdee08'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:00.596887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1381/2743] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:00 2022] DELETE /identity/v3/users/02f52107d00746e08b98de2331fdee08 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:01.039268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:01.040968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:01.061114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02332-cc1e-4ba8-8ba5-c17ed42ab0da service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:01.103465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:01.122193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:01.122841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.123147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.123442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] PATH_INFO: `/v3/projects/07cc4c6bf07f4f479e6ec525186fcd9e` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.124623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=07cc4c6bf07f4f479e6ec525186fcd9e)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:01.126650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:01.245842 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07cc4c6bf07f4f479e6ec525186fcd9e', 'request_id': 'req-6928f87a-d37d-4212-b236-87490709ff0c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:01.270517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:01.273876 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:01.362703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07cc4c6bf07f4f479e6ec525186fcd9e', 'request_id': 'req-6928f87a-d37d-4212-b236-87490709ff0c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:01.373778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07cc4c6bf07f4f479e6ec525186fcd9e', 'request_id': 'req-6928f87a-d37d-4212-b236-87490709ff0c'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:01.388855 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] The token cache is being invalidate because project 07cc4c6bf07f4f479e6ec525186fcd9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:01.389535 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:01.390325 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-6928f87a-d37d-4212-b236-87490709ff0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:01.392968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1363/2744] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:01 2022] DELETE /identity/v3/projects/07cc4c6bf07f4f479e6ec525186fcd9e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:01.402231 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d42ed-c1cb-4e73-a4c4-8012a58e9b5e None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.402692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d42ed-c1cb-4e73-a4c4-8012a58e9b5e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.403364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d42ed-c1cb-4e73-a4c4-8012a58e9b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.428897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-9f1d42ed-c1cb-4e73-a4c4-8012a58e9b5e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:01.430377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-9f1d42ed-c1cb-4e73-a4c4-8012a58e9b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:01.458339 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84312ae1-1e32-4f93-a295-40c20f9fa74a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:01.468887 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:01.504692 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1364/2745] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:13:01.518498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96373046-807b-464c-a7e2-ad98d8464035 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.518780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96373046-807b-464c-a7e2-ad98d8464035 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.519175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96373046-807b-464c-a7e2-ad98d8464035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.545889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-96373046-807b-464c-a7e2-ad98d8464035 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:01.547538 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-96373046-807b-464c-a7e2-ad98d8464035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:01.557612 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1365/2746] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:01.561940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96373046-807b-464c-a7e2-ad98d8464035 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:01.562227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:01.563126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.563617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.564120 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.567202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:01.573172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96373046-807b-464c-a7e2-ad98d8464035 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:01.578638 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:01.579643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.580000 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.581128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.581637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:01.582793 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:01.586144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:01.588814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:01.590863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:01.606107 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1382/2747] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:13:01.606257 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1366/2748] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:13:01.606964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:01.618271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce72bab-1e47-4394-b83d-46567a4b358a service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:01.624916 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:01.625565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.626101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.630548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.630548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:01.630548 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:01.736672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1383/2749] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:01 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:13:01.741704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:01.752020 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c7b4d-2069-4cdc-a3b5-d17d3aef81af admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:01.807389 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:01.818914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:01.819541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.819819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.820089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.820895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:01.826652 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:01.912123 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1367/2750] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:01 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:01.916478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:01.928225 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789570f-d065-42f0-bc15-9afd7c493bd0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:01.942691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:01.942691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.942691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.942691 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.943094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:01.943094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:01.947847 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1384/2751] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:01.951976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:01.964349 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392df11d-9e7f-408d-8fcb-e0b2db884488 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:01.971050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571e2f9-8506-428b-a556-751a4303d26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:01.971952 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e2f9-8506-428b-a556-751a4303d26c admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:01.973066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e2f9-8506-428b-a556-751a4303d26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:01.973801 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e2f9-8506-428b-a556-751a4303d26c admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:01.976985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571e2f9-8506-428b-a556-751a4303d26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:01.976985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571e2f9-8506-428b-a556-751a4303d26c admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:01.988156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1368/2752] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:01.995493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:02.011168 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f10de-4b56-45df-8763-dba5c6538ab9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:02.016478 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:02.017141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:02.017418 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:02.017704 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] PATH_INFO: `/v3/projects/b3c0478cb8c143d4a62f13a0be951ed1/users/9e5b85a2845c4a9592b1a855eb41ff88/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:02.035774 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c0478cb8c143d4a62f13a0be951ed1, user_id=9e5b85a2845c4a9592b1a855eb41ff88, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:02.038589 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:02.137379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1385/2753] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:01 2022] PUT /identity/v3/projects/b3c0478cb8c143d4a62f13a0be951ed1/users/9e5b85a2845c4a9592b1a855eb41ff88/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:02.150322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:02.150844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:02.151063 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:02.187682 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] MFA Rules not processed for user `9e5b85a2845c4a9592b1a855eb41ff88`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:02.188770 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:02.261700 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1369/2754] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:13:02.277112 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:02.279272 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:02.370051 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9669b-8d1f-484d-b8f0-86327ac7d143 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:02.374417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04084ffe1b0548fea75ea144977e4f15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:02.375096 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:02.375460 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:02.375794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:02.387267 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:02.388811 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:02.400393 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1386/2755] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:04.224966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:04.227170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:04.303074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b7caa-a84b-49a1-8db3-518a53cd1ee6 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:04.307913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:04.308556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:04.308863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:04.309340 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] PATH_INFO: `/v3/users/c27662f0f0fe47ac9445b65599186ac4` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:04.310568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27662f0f0fe47ac9445b65599186ac4)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:04.312473 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:04.616920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27662f0f0fe47ac9445b65599186ac4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:04.617605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27662f0f0fe47ac9445b65599186ac4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:04.624653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27662f0f0fe47ac9445b65599186ac4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:04.630081 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:04.638355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27662f0f0fe47ac9445b65599186ac4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:04.639070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27662f0f0fe47ac9445b65599186ac4'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:04.726772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1370/2756] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:04 2022] DELETE /identity/v3/users/c27662f0f0fe47ac9445b65599186ac4 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:05.011252 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:05.022586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25d40d-48f6-4450-aa9f-9f7f95cfa07d service glance] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:05.075674 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:05.092810 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:05.093449 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:05.093732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:05.094035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] PATH_INFO: `/v3/projects/90761a2b55bd4ba18ad11f0938269fbc` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:05.095170 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90761a2b55bd4ba18ad11f0938269fbc)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:05.097855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:05.230361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90761a2b55bd4ba18ad11f0938269fbc', 'request_id': 'req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.317171 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90761a2b55bd4ba18ad11f0938269fbc', 'request_id': 'req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.336464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90761a2b55bd4ba18ad11f0938269fbc', 'request_id': 'req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.350702 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] The token cache is being invalidate because project 90761a2b55bd4ba18ad11f0938269fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:05.351041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.351760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.354007 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1387/2757] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:05 2022] DELETE /identity/v3/projects/90761a2b55bd4ba18ad11f0938269fbc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:05.357897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:05.359779 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:05.375678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d740f1-e334-4a3c-9450-bea453e2a293 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:05.456222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:05.457511 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:05.457833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:05.458228 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] PATH_INFO: `/v3/users/369a398d1cb04c8993fb442912b49ec5` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:05.459498 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=369a398d1cb04c8993fb442912b49ec5)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:05.461409 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:05.641298 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '369a398d1cb04c8993fb442912b49ec5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.642018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '369a398d1cb04c8993fb442912b49ec5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.649819 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '369a398d1cb04c8993fb442912b49ec5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.668056 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '369a398d1cb04c8993fb442912b49ec5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.668808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0ea3423-85dc-4ae3-b334-5d63dc1694fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '369a398d1cb04c8993fb442912b49ec5'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:05.755294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1371/2758] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:05 2022] DELETE /identity/v3/users/369a398d1cb04c8993fb442912b49ec5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:06.036379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:06.037849 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:06.051318 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb21b9-bf4d-4856-b53b-6ee8c4d25125 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:06.115720 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:06.116349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.116626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.116894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] PATH_INFO: `/v3/projects/12c05dbe3dda47b99feda2f7dd9283e5` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.118061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c05dbe3dda47b99feda2f7dd9283e5)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:06.119956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:06.231878 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c05dbe3dda47b99feda2f7dd9283e5', 'request_id': 'req-ffe49244-c085-4fba-905a-9f5ab428f76a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:06.319733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c05dbe3dda47b99feda2f7dd9283e5', 'request_id': 'req-ffe49244-c085-4fba-905a-9f5ab428f76a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:06.326722 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c05dbe3dda47b99feda2f7dd9283e5', 'request_id': 'req-ffe49244-c085-4fba-905a-9f5ab428f76a'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:06.337863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] The token cache is being invalidate because project 12c05dbe3dda47b99feda2f7dd9283e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:06.338149 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:06.338781 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffe49244-c085-4fba-905a-9f5ab428f76a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:06.340279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1388/2759] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:06 2022] DELETE /identity/v3/projects/12c05dbe3dda47b99feda2f7dd9283e5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:06.346084 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ec9e2-c7fc-420e-94f6-614eec6fb257 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.346368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ec9e2-c7fc-420e-94f6-614eec6fb257 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.346637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ec9e2-c7fc-420e-94f6-614eec6fb257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.366696 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-745ec9e2-c7fc-420e-94f6-614eec6fb257 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:06.367895 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-745ec9e2-c7fc-420e-94f6-614eec6fb257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:06.418296 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:06.427994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1372/2760] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:06.436911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4e64-f5e0-4342-ba13-c3cc71655de2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.437328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4e64-f5e0-4342-ba13-c3cc71655de2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.437632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4e64-f5e0-4342-ba13-c3cc71655de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.445301 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:06.470295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-79ac4e64-f5e0-4342-ba13-c3cc71655de2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:06.471601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-79ac4e64-f5e0-4342-ba13-c3cc71655de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:06.484398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1389/2761] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:06.485828 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ec9e2-c7fc-420e-94f6-614eec6fb257 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:06.501780 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ec9e2-c7fc-420e-94f6-614eec6fb257 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:06.507137 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:06.507844 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.508350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.508660 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.509693 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:06.509836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:06.512060 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:06.525782 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1373/2762] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:13:06.529977 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac4e64-f5e0-4342-ba13-c3cc71655de2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:06.540068 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac4e64-f5e0-4342-ba13-c3cc71655de2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:06.545164 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:06.545787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.551842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.551842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.551842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:06.551842 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:06.678043 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1390/2763] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:06 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:06.682341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:06.692824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d84a3-6744-4408-8048-f2a08272e371 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:06.752052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:06.752650 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.752930 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.753204 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.754367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:06.756272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:06.853122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1374/2764] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:06 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:06.857192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:06.868932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33f8f-ada5-4ff4-bc96-8a03e3d398e6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:06.874807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:06.875435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.876135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.876135 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.877800 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:06.879250 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:06.890905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1391/2765] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:06.893759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:06.905385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14027e8-b902-47e8-9de3-a56a16d917bf admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:06.911429 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee448fd-161a-407e-bb1f-2b9bda36b4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:06.912076 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee448fd-161a-407e-bb1f-2b9bda36b4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.912322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee448fd-161a-407e-bb1f-2b9bda36b4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.912704 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee448fd-161a-407e-bb1f-2b9bda36b4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.913288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee448fd-161a-407e-bb1f-2b9bda36b4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:06.914942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee448fd-161a-407e-bb1f-2b9bda36b4f9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:06.928729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1375/2766] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:06.931446 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:06.942422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab9d2-a20c-40ba-88c7-bb6505d94fee admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:06.947221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:06.947882 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:06.948169 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:06.948447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] PATH_INFO: `/v3/projects/10065b494c084e3caf0f4b4f36e13c73/users/f134ec0b2543442db5ccdc4377326277/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:06.967045 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10065b494c084e3caf0f4b4f36e13c73, user_id=f134ec0b2543442db5ccdc4377326277, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:06.969884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:07.087459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1392/2767] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:06 2022] PUT /identity/v3/projects/10065b494c084e3caf0f4b4f36e13c73/users/f134ec0b2543442db5ccdc4377326277/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:07.098658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f537-6459-4d56-adff-21d63c2a6676 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:07.098942 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f537-6459-4d56-adff-21d63c2a6676 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:07.099211 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f537-6459-4d56-adff-21d63c2a6676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:07.130156 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-4139f537-6459-4d56-adff-21d63c2a6676 None None] MFA Rules not processed for user `f134ec0b2543442db5ccdc4377326277`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:07.131549 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-4139f537-6459-4d56-adff-21d63c2a6676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:07.211852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1376/2768] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:07.223323 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:07.225320 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:07.289785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a07be5-aafb-4702-94cd-f4082f14822d admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:07.293869 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc655939-2f5b-4a23-8fa8-1fd417317d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:07.294497 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc655939-2f5b-4a23-8fa8-1fd417317d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:07.294839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc655939-2f5b-4a23-8fa8-1fd417317d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:07.295160 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc655939-2f5b-4a23-8fa8-1fd417317d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:07.306633 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc655939-2f5b-4a23-8fa8-1fd417317d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:07.308183 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc655939-2f5b-4a23-8fa8-1fd417317d8e service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:07.318881 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1393/2769] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:07.340739 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f537-6459-4d56-adff-21d63c2a6676 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:07.399920 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:07.413402 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f537-6459-4d56-adff-21d63c2a6676 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:07.418299 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:07.418931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:07.419210 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:07.419486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] PATH_INFO: `/v3/users/f134ec0b2543442db5ccdc4377326277` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:07.420600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f134ec0b2543442db5ccdc4377326277)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:07.422586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:07.585795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f134ec0b2543442db5ccdc4377326277'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:07.586385 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f134ec0b2543442db5ccdc4377326277'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:07.599107 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f134ec0b2543442db5ccdc4377326277'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:07.620550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f134ec0b2543442db5ccdc4377326277'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:07.621293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f134ec0b2543442db5ccdc4377326277'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:07.685727 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1377/2770] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:07 2022] DELETE /identity/v3/users/f134ec0b2543442db5ccdc4377326277 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:07.690368 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f9a90-bc82-4560-a60d-dd786cc90806 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:07.690935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f9a90-bc82-4560-a60d-dd786cc90806 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:07.691437 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f9a90-bc82-4560-a60d-dd786cc90806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:07.711697 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-176f9a90-bc82-4560-a60d-dd786cc90806 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:07.712841 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-176f9a90-bc82-4560-a60d-dd786cc90806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:07.783387 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1394/2771] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:07.793977 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:07.795581 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:07.859364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8849ac-8edd-4c11-8b6f-c5499fa1074d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:07.863517 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:07.864149 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:07.864482 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:07.864759 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:07.875633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:07.877078 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:07.888185 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1378/2772] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:07.974319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:07.974611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:07.974954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:07.997668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] MFA Rules not processed for user `81634611141f4909a52ac46d5164f17f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:07.999088 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:08.041860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:08.061792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1395/2773] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:08.392839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:08.394831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:08.408463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb1ca-a130-4a68-8018-c536a2d88ded service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:08.412726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b6c92-3e34-48f7-b3b3-a28f01f09bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:08.413388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b6c92-3e34-48f7-b3b3-a28f01f09bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:08.413681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b6c92-3e34-48f7-b3b3-a28f01f09bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:08.413985 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b6c92-3e34-48f7-b3b3-a28f01f09bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:08.415776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-774b6c92-3e34-48f7-b3b3-a28f01f09bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:08.427531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b6c92-3e34-48f7-b3b3-a28f01f09bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:08.429021 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b6c92-3e34-48f7-b3b3-a28f01f09bd4 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:08.448265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:08.450287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:08.453475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1379/2774] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:08.471082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e39a4-c86a-465b-8750-278db63b021e None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:08.491216 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:08.492215 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:08.492727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:08.493202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] PATH_INFO: `/v3/projects/10065b494c084e3caf0f4b4f36e13c73` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:08.498757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10065b494c084e3caf0f4b4f36e13c73)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:08.501529 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:08.718462 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10065b494c084e3caf0f4b4f36e13c73', 'request_id': 'req-d56f7376-0bba-4640-9ea2-6712e7ebdd47'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:08.836394 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10065b494c084e3caf0f4b4f36e13c73', 'request_id': 'req-d56f7376-0bba-4640-9ea2-6712e7ebdd47'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:08.850434 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10065b494c084e3caf0f4b4f36e13c73', 'request_id': 'req-d56f7376-0bba-4640-9ea2-6712e7ebdd47'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:08.871784 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] The token cache is being invalidate because project 10065b494c084e3caf0f4b4f36e13c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:08.877664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:08.877664 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-d56f7376-0bba-4640-9ea2-6712e7ebdd47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:08.879192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1396/2775] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:08 2022] DELETE /identity/v3/projects/10065b494c084e3caf0f4b4f36e13c73 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:08.880280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92207a75-aef8-41a6-a775-2b3ebd8babbe None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:08.880628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92207a75-aef8-41a6-a775-2b3ebd8babbe None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:08.881062 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92207a75-aef8-41a6-a775-2b3ebd8babbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:08.902678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-92207a75-aef8-41a6-a775-2b3ebd8babbe None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:08.903983 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-92207a75-aef8-41a6-a775-2b3ebd8babbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:08.969928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1380/2776] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:13:08.978076 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44bdd-cfb9-4497-b172-81838724bb23 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:08.978279 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44bdd-cfb9-4497-b172-81838724bb23 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:08.978578 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44bdd-cfb9-4497-b172-81838724bb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:08.999562 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-4dc44bdd-cfb9-4497-b172-81838724bb23 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:09.000795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-4dc44bdd-cfb9-4497-b172-81838724bb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:09.009361 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1397/2777] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:09.013721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92207a75-aef8-41a6-a775-2b3ebd8babbe None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:09.018869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:09.026655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92207a75-aef8-41a6-a775-2b3ebd8babbe None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:09.031550 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:09.032184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.032459 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.032725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.033367 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:09.033665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:09.035558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:09.048226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1381/2778] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:13:09.052981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc44bdd-cfb9-4497-b172-81838724bb23 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:09.062956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc44bdd-cfb9-4497-b172-81838724bb23 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:09.068594 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:09.069218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.069525 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.069846 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.070518 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:09.072480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:09.273894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1398/2779] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:09 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:09.282703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:09.293524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791169ba-2b91-483f-b36c-d4e8edbece82 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:09.354431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:09.355065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.355339 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.355609 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.356265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:09.358175 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:09.477377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1382/2780] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:09 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:13:09.481652 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:09.493040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d0a75-d093-4f35-ae58-d3074df00e77 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:09.497646 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:09.498208 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.498487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.498764 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.504055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:09.504055 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:09.515654 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1399/2781] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:09.520678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:09.534533 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b0850-ca8a-4d0b-bb20-fdfbdf5d1d26 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:09.539189 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e280f7c-cbdb-4404-88f8-d15f162871e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:09.539805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280f7c-cbdb-4404-88f8-d15f162871e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.540086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280f7c-cbdb-4404-88f8-d15f162871e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.540360 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280f7c-cbdb-4404-88f8-d15f162871e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.541011 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e280f7c-cbdb-4404-88f8-d15f162871e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:09.542640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e280f7c-cbdb-4404-88f8-d15f162871e2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:09.556761 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1383/2782] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:09.561218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:09.572001 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394b37a-905f-4ee2-8ab8-872f607ef1d1 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:09.576670 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:09.577316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.577688 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.578019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] PATH_INFO: `/v3/projects/a26d0a86c9d54012b058c7127cc71ecd/users/833622fc7358412ba4c0c092ee8bbb61/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.594173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26d0a86c9d54012b058c7127cc71ecd, user_id=833622fc7358412ba4c0c092ee8bbb61, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:09.597087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:09.657766 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1400/2783] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:09 2022] PUT /identity/v3/projects/a26d0a86c9d54012b058c7127cc71ecd/users/833622fc7358412ba4c0c092ee8bbb61/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:09.669592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.671124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.671124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.705702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] MFA Rules not processed for user `833622fc7358412ba4c0c092ee8bbb61`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:09.707059 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:09.779147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1384/2784] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:09.790716 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:09.792464 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:09.852289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90be2-0db7-4a8b-893a-aff264bafcee admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:09.860106 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:09.860738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:09.861023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:09.861358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:09.871401 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:09.872974 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:09.886321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1401/2785] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:10.293951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:10.295649 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:10.484853 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f475db-de2a-403c-b573-b9282d043e75 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:10.489265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c4b000-6bda-49b7-ba22-f53b98c492f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:10.489929 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4b000-6bda-49b7-ba22-f53b98c492f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:10.490209 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4b000-6bda-49b7-ba22-f53b98c492f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:10.490481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c4b000-6bda-49b7-ba22-f53b98c492f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:10.492206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-d9c4b000-6bda-49b7-ba22-f53b98c492f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:10.502578 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:10.560375 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4b000-6bda-49b7-ba22-f53b98c492f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:10.561843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c4b000-6bda-49b7-ba22-f53b98c492f1 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:10.576241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1385/2786] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:12.276760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:12.279360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:12.312624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:12.411222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84a748-37b5-4fa6-8f5c-c7fb3057c855 service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:12.416624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:12.417333 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:12.423451 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:12.424185 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:12.426985 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:12.439477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:12.441884 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:12.452978 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1402/2787] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:12.763223 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed05642-e8f6-4fd0-abb9-83a7d8fd1b67 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:12.763800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed05642-e8f6-4fd0-abb9-83a7d8fd1b67 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:12.764077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed05642-e8f6-4fd0-abb9-83a7d8fd1b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:12.792628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7ed05642-e8f6-4fd0-abb9-83a7d8fd1b67 None None] MFA Rules not processed for user `833622fc7358412ba4c0c092ee8bbb61`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:12.794270 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7ed05642-e8f6-4fd0-abb9-83a7d8fd1b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:12.803277 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1386/2788] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:13:21.514528 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:21.515960 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:21.535159 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea1d1e-8c10-4b95-8915-ae1497b618f4 service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:21.541057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:21.541927 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:21.542681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:21.543057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] PATH_INFO: `/v3/users/4690104f5bea480d87b7c43488182441` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:21.544414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4690104f5bea480d87b7c43488182441)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:21.554839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:21.750238 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4690104f5bea480d87b7c43488182441'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:21.751070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4690104f5bea480d87b7c43488182441'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:21.758306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4690104f5bea480d87b7c43488182441'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:21.786695 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4690104f5bea480d87b7c43488182441'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:21.787524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4690104f5bea480d87b7c43488182441'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:21.855713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1403/2789] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:21 2022] DELETE /identity/v3/users/4690104f5bea480d87b7c43488182441 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:22.142994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed05642-e8f6-4fd0-abb9-83a7d8fd1b67 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:22.157995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed05642-e8f6-4fd0-abb9-83a7d8fd1b67 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:22.253112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:22.253856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:22.254139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:22.254575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] PATH_INFO: `/v3/projects/9788b5b66a0c452cb39b263ca30284e0` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:22.256018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9788b5b66a0c452cb39b263ca30284e0)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:22.258172 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:22.414222 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9788b5b66a0c452cb39b263ca30284e0', 'request_id': 'req-84910439-5dee-4ec7-ad6b-5f192c38ba12'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.515605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9788b5b66a0c452cb39b263ca30284e0', 'request_id': 'req-84910439-5dee-4ec7-ad6b-5f192c38ba12'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.523709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9788b5b66a0c452cb39b263ca30284e0', 'request_id': 'req-84910439-5dee-4ec7-ad6b-5f192c38ba12'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.545293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] The token cache is being invalidate because project 9788b5b66a0c452cb39b263ca30284e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:22.545602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.546683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.548556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1387/2790] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:22 2022] DELETE /identity/v3/projects/9788b5b66a0c452cb39b263ca30284e0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:22.553649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:22.555507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:22.570161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a0234-1bcb-4539-a920-f2d2f1e0ef0b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:22.639777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:22.654026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:22.655060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:22.655347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:22.656000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] PATH_INFO: `/v3/users/3372ac9db1e649a898e15c3bf664fc3b` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:22.657190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3372ac9db1e649a898e15c3bf664fc3b)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:22.659827 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:22.882081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3372ac9db1e649a898e15c3bf664fc3b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.882763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3372ac9db1e649a898e15c3bf664fc3b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.889863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3372ac9db1e649a898e15c3bf664fc3b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.902950 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3372ac9db1e649a898e15c3bf664fc3b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.908420 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3372ac9db1e649a898e15c3bf664fc3b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:22.990101 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1404/2791] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:22 2022] DELETE /identity/v3/users/3372ac9db1e649a898e15c3bf664fc3b => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:23.140928 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:23.142453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:23.166454 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84910439-5dee-4ec7-ad6b-5f192c38ba12 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:23.196781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:23.227232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:23.227871 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:23.228152 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:23.228426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] PATH_INFO: `/v3/users/9e5b85a2845c4a9592b1a855eb41ff88` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:23.229556 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5b85a2845c4a9592b1a855eb41ff88)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:23.231496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:23.264643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:23.266140 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:23.280099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a6057-2331-402c-a3a4-e6aff28dd06a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:23.287356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:23.288234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:23.288517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:23.288796 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] PATH_INFO: `/v3/projects/686c7aa2d3384c8a890065be02c5078f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:23.290010 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686c7aa2d3384c8a890065be02c5078f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:23.291992 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:23.434858 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686c7aa2d3384c8a890065be02c5078f', 'request_id': 'req-a0b0a83c-8357-4bf6-8b82-bac8f277d185'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.436883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5b85a2845c4a9592b1a855eb41ff88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.438025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5b85a2845c4a9592b1a855eb41ff88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.445805 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5b85a2845c4a9592b1a855eb41ff88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.458740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5b85a2845c4a9592b1a855eb41ff88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.459406 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5b85a2845c4a9592b1a855eb41ff88'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.527886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686c7aa2d3384c8a890065be02c5078f', 'request_id': 'req-a0b0a83c-8357-4bf6-8b82-bac8f277d185'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.537692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686c7aa2d3384c8a890065be02c5078f', 'request_id': 'req-a0b0a83c-8357-4bf6-8b82-bac8f277d185'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.547414 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] The token cache is being invalidate because project 686c7aa2d3384c8a890065be02c5078f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:23.547711 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.548701 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-a0b0a83c-8357-4bf6-8b82-bac8f277d185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:23.551138 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1405/2792] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:23 2022] DELETE /identity/v3/projects/686c7aa2d3384c8a890065be02c5078f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:23.590012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1388/2793] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:23 2022] DELETE /identity/v3/users/9e5b85a2845c4a9592b1a855eb41ff88 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:23.594833 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:23.600468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:23.600468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:23.661910 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:23.663382 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:23.735440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:23.760070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1406/2794] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:23.774363 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:23.776547 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:23.898427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dbf29-03df-4f96-8686-465a94123104 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:23.901969 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2532d-c2ca-4b1f-871a-a4f727b0c8bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:23.902574 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2532d-c2ca-4b1f-871a-a4f727b0c8bd service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:23.902852 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2532d-c2ca-4b1f-871a-a4f727b0c8bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:23.903124 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2532d-c2ca-4b1f-871a-a4f727b0c8bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:23.912909 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2532d-c2ca-4b1f-871a-a4f727b0c8bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:23.914379 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2532d-c2ca-4b1f-871a-a4f727b0c8bd service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:23.923918 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1389/2795] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:24.350860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:24.352286 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:24.364595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe1d7-8dc2-4a29-88cc-2bbb887a36b4 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:24.369200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:24.369993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:24.370271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:24.381182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] PATH_INFO: `/v3/projects/b3c0478cb8c143d4a62f13a0be951ed1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:24.381182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c0478cb8c143d4a62f13a0be951ed1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:24.381182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:24.498337 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c0478cb8c143d4a62f13a0be951ed1', 'request_id': 'req-f18d3278-610e-47ab-9f5e-0e7c97976379'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:24.578879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c0478cb8c143d4a62f13a0be951ed1', 'request_id': 'req-f18d3278-610e-47ab-9f5e-0e7c97976379'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:24.586487 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c0478cb8c143d4a62f13a0be951ed1', 'request_id': 'req-f18d3278-610e-47ab-9f5e-0e7c97976379'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:24.598896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] The token cache is being invalidate because project b3c0478cb8c143d4a62f13a0be951ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:24.599484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:24.600002 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-f18d3278-610e-47ab-9f5e-0e7c97976379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:24.601492 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1407/2796] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:24 2022] DELETE /identity/v3/projects/b3c0478cb8c143d4a62f13a0be951ed1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:24.606523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd77d-f563-482d-9e0a-aba8eb89038c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:24.606970 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd77d-f563-482d-9e0a-aba8eb89038c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:24.607066 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd77d-f563-482d-9e0a-aba8eb89038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:24.626555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-93ccd77d-f563-482d-9e0a-aba8eb89038c None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:24.628353 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-93ccd77d-f563-482d-9e0a-aba8eb89038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:24.661950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:24.694605 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1390/2797] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:24.703455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcf10b-0592-4f66-8ebe-10591f2639ba None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:24.703732 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcf10b-0592-4f66-8ebe-10591f2639ba None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:24.704012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcf10b-0592-4f66-8ebe-10591f2639ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:24.726218 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fedcf10b-0592-4f66-8ebe-10591f2639ba None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:24.727445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fedcf10b-0592-4f66-8ebe-10591f2639ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:24.735570 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1408/2798] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:24.742748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd77d-f563-482d-9e0a-aba8eb89038c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:24.754502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd77d-f563-482d-9e0a-aba8eb89038c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:24.759335 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:24.759955 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:24.760237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:24.760513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:24.761170 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:24.761475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:24.763117 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:24.775585 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1391/2799] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:13:24.780018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcf10b-0592-4f66-8ebe-10591f2639ba None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:24.791560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcf10b-0592-4f66-8ebe-10591f2639ba None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:24.803245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:24.803245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:24.803245 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:24.803651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:24.804403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:24.806405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:24.929204 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1409/2800] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:24 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:24.930192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:24.942122 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8677e-bf49-4b9e-b868-eae9f33723d8 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:25.013373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:25.014040 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:25.014319 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:25.014588 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:25.015242 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:25.017235 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:25.117470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1392/2801] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:24 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:25.129375 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:25.140813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344c9a5-3d0a-4b19-8c12-2574fb548d17 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:25.147585 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:25.148326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:25.148690 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:25.149033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:25.154349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:25.154349 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:25.165803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1410/2802] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:25.169440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:25.180998 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5982f-d744-4c50-aee7-1b3f84a6c26d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:25.186442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675962aa-d52a-4644-a7dd-23e308bed6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:25.187154 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675962aa-d52a-4644-a7dd-23e308bed6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:25.187496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675962aa-d52a-4644-a7dd-23e308bed6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:25.187840 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675962aa-d52a-4644-a7dd-23e308bed6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:25.188552 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675962aa-d52a-4644-a7dd-23e308bed6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:25.190467 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675962aa-d52a-4644-a7dd-23e308bed6c3 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:25.201569 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1393/2803] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:25.205655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:25.216097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f278b-79e2-4a26-8e41-bc1389f79580 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:25.220657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:25.221296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:25.222173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:25.222536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] PATH_INFO: `/v3/projects/db23062a20404e9eb62e900e0bc745c3/users/071147f90cf1405397304fd3b3236911/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:25.237539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db23062a20404e9eb62e900e0bc745c3, user_id=071147f90cf1405397304fd3b3236911, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:25.240293 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:25.339980 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1411/2804] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:25 2022] PUT /identity/v3/projects/db23062a20404e9eb62e900e0bc745c3/users/071147f90cf1405397304fd3b3236911/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:25.351856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20386b2e-db8e-4b16-b8b0-ee6c0d43797c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:25.352045 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20386b2e-db8e-4b16-b8b0-ee6c0d43797c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:25.352316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20386b2e-db8e-4b16-b8b0-ee6c0d43797c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:25.381276 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-20386b2e-db8e-4b16-b8b0-ee6c0d43797c None None] MFA Rules not processed for user `071147f90cf1405397304fd3b3236911`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:25.382521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-20386b2e-db8e-4b16-b8b0-ee6c0d43797c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:25.448120 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1394/2805] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:25.459944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:25.461677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:25.548289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dd831-9b4c-439f-991c-3158a74fd4d0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:25.552838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ea32e-2d10-469e-b315-10cb76cce97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:25.553459 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ea32e-2d10-469e-b315-10cb76cce97b service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:25.553755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ea32e-2d10-469e-b315-10cb76cce97b service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:25.554025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ea32e-2d10-469e-b315-10cb76cce97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:25.560195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:25.562352 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ea32e-2d10-469e-b315-10cb76cce97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:25.563761 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ea32e-2d10-469e-b315-10cb76cce97b service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:25.572945 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1412/2806] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:35.445241 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:35.627891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20386b2e-db8e-4b16-b8b0-ee6c0d43797c None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:35.658143 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:35.692822 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20386b2e-db8e-4b16-b8b0-ee6c0d43797c None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:35.697451 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:35.698114 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:35.698393 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:35.698683 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] PATH_INFO: `/v3/users/071147f90cf1405397304fd3b3236911` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:35.700193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=071147f90cf1405397304fd3b3236911)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:35.702138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:35.871530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071147f90cf1405397304fd3b3236911'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:35.872192 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071147f90cf1405397304fd3b3236911'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:35.879670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071147f90cf1405397304fd3b3236911'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:35.891531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071147f90cf1405397304fd3b3236911'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:35.892166 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071147f90cf1405397304fd3b3236911'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:35.982067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1395/2807] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:35 2022] DELETE /identity/v3/users/071147f90cf1405397304fd3b3236911 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:35.986500 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:35.986789 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:35.987066 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:36.006662 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:36.007866 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:36.065767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1413/2808] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:36.075709 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:36.077381 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:36.134778 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20396ac3-4256-4bf3-bb98-14f4cb6f9f7d admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:36.138440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f81043-4376-4c81-b4cb-81340a8a266c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:36.139041 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f81043-4376-4c81-b4cb-81340a8a266c service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:36.139322 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f81043-4376-4c81-b4cb-81340a8a266c service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:36.139594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f81043-4376-4c81-b4cb-81340a8a266c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:36.148836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f81043-4376-4c81-b4cb-81340a8a266c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:36.150320 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f81043-4376-4c81-b4cb-81340a8a266c service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:36.160525 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1396/2809] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:13:36.613727 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:36.615587 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:36.640186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e918ca94-55f3-4008-b3f8-9ef3803849bf None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:36.650639 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:36.651296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:36.651581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:36.651859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] PATH_INFO: `/v3/projects/db23062a20404e9eb62e900e0bc745c3` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:36.653020 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db23062a20404e9eb62e900e0bc745c3)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:36.655021 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:36.782607 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db23062a20404e9eb62e900e0bc745c3', 'request_id': 'req-b8ad14c0-8717-4052-9845-9feed57d5014'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:36.875603 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db23062a20404e9eb62e900e0bc745c3', 'request_id': 'req-b8ad14c0-8717-4052-9845-9feed57d5014'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:36.888125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db23062a20404e9eb62e900e0bc745c3', 'request_id': 'req-b8ad14c0-8717-4052-9845-9feed57d5014'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:36.907515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] The token cache is being invalidate because project db23062a20404e9eb62e900e0bc745c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:36.913435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:36.913435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b8ad14c0-8717-4052-9845-9feed57d5014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:36.913435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1414/2810] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:36 2022] DELETE /identity/v3/projects/db23062a20404e9eb62e900e0bc745c3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:36.915494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d71b-2cfb-43c7-872c-d3501c8a60be None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:36.915987 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d71b-2cfb-43c7-872c-d3501c8a60be None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:36.916094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d71b-2cfb-43c7-872c-d3501c8a60be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:36.943178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-2f95d71b-2cfb-43c7-872c-d3501c8a60be None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:36.944425 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-2f95d71b-2cfb-43c7-872c-d3501c8a60be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:36.963196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:37.008728 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1397/2811] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:37.016687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bc05f-2f09-4663-b674-c07326e6b964 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.016905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bc05f-2f09-4663-b674-c07326e6b964 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.017184 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bc05f-2f09-4663-b674-c07326e6b964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.039343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-740bc05f-2f09-4663-b674-c07326e6b964 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:37.040477 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-740bc05f-2f09-4663-b674-c07326e6b964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:37.049268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1415/2812] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:37.054740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95d71b-2cfb-43c7-872c-d3501c8a60be None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:37.064766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95d71b-2cfb-43c7-872c-d3501c8a60be None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:37.071297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:37.071831 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.072112 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.072386 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.073043 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:37.073368 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:37.075191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:37.088269 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1398/2813] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:13:37.092877 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bc05f-2f09-4663-b674-c07326e6b964 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:37.102278 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bc05f-2f09-4663-b674-c07326e6b964 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:37.107006 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:37.107677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.108018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.108378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.109017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:37.110911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:37.219415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1416/2814] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:37 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:13:37.223787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:37.238094 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ce221-91b7-4dba-a914-15a3adf89326 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:37.345863 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:37.346493 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.346874 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.348126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.348388 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:37.350496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:37.464995 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1399/2815] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:37 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:37.470813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:37.483004 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74cf3e-aa25-48fa-a703-5bd412e3dfed admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:37.492290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:37.493601 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.494085 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.501265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.501265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:37.501265 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:37.511535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1417/2816] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:37.515726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:37.526729 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d39ae1-a875-4da5-94ce-d5c4b5f4af39 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:37.543507 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8cc7d-d716-4259-8c58-4f4d19f4e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:37.544243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8cc7d-d716-4259-8c58-4f4d19f4e12a admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.544582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8cc7d-d716-4259-8c58-4f4d19f4e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.544914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8cc7d-d716-4259-8c58-4f4d19f4e12a admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.545631 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8cc7d-d716-4259-8c58-4f4d19f4e12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:37.547394 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8cc7d-d716-4259-8c58-4f4d19f4e12a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:37.559427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1400/2817] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:13:37.563579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:37.573863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464160e-1003-40db-94ac-57c75007c0f6 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:37.578432 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:37.579070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.579347 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.579620 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] PATH_INFO: `/v3/projects/a5d396f1d5c14e179ab56809f912f1dc/users/6218b07ed7864e2589de6eff3fd1b08d/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.594602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d396f1d5c14e179ab56809f912f1dc, user_id=6218b07ed7864e2589de6eff3fd1b08d, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:37.598384 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:37.664687 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1418/2818] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:37 2022] PUT /identity/v3/projects/a5d396f1d5c14e179ab56809f912f1dc/users/6218b07ed7864e2589de6eff3fd1b08d/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:37.670126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0aa16c-aa4a-49cd-b7bd-80192a1defa9 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.670563 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0aa16c-aa4a-49cd-b7bd-80192a1defa9 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.670846 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0aa16c-aa4a-49cd-b7bd-80192a1defa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.691927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-7b0aa16c-aa4a-49cd-b7bd-80192a1defa9 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:37.693139 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-7b0aa16c-aa4a-49cd-b7bd-80192a1defa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:37.759371 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1401/2819] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 13:13:37.774263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:37.774263 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:37.795283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:37.831870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375a01b-53d3-44c3-bc2d-cc533d16e1bf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:37.836484 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:37.837100 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:37.837558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:37.837712 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:37.847816 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:37.849713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:37.859839 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1419/2820] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:39.173226 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c320ded-cc66-4324-be7a-2ce4f4becf01 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:39.173688 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c320ded-cc66-4324-be7a-2ce4f4becf01 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:39.174006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c320ded-cc66-4324-be7a-2ce4f4becf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:39.215236 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-8c320ded-cc66-4324-be7a-2ce4f4becf01 None None] MFA Rules not processed for user `6218b07ed7864e2589de6eff3fd1b08d`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:39.216594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8c320ded-cc66-4324-be7a-2ce4f4becf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:39.246689 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:39.297551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1402/2821] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:39.309523 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:39.311886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:39.372737 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c9368-14f2-4f5f-8905-29d4363b1301 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:39.377193 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:39.377843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:39.378898 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:39.379269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:39.387834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:39.389271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:39.398430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1420/2822] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:40.053760 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23f6f-6607-4739-ba2b-71ed17e2b06d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:40.054082 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23f6f-6607-4739-ba2b-71ed17e2b06d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:40.054352 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23f6f-6607-4739-ba2b-71ed17e2b06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:40.083440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-53a23f6f-6607-4739-ba2b-71ed17e2b06d None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:40.084725 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-53a23f6f-6607-4739-ba2b-71ed17e2b06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:40.140044 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1403/2823] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:13:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:40.314047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:40.315463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:40.384348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1024fa-f41f-4f67-9727-b987d8274b1e service cinder] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:40.389317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:40.389944 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:40.390223 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:40.390494 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:40.399012 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:40.400576 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:40.410023 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1421/2824] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:13:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:13:41.300566 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60627918-1634-41bf-b3ec-0eddb0641b5f None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:41.301085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60627918-1634-41bf-b3ec-0eddb0641b5f None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:41.301085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60627918-1634-41bf-b3ec-0eddb0641b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:41.321013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-60627918-1634-41bf-b3ec-0eddb0641b5f None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:41.322229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-60627918-1634-41bf-b3ec-0eddb0641b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:41.329426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1404/2825] 10.222.0.72 () {58 vars in 967 bytes} [Tue Sep 13 13:13:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:41.342317 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:41.352964 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68bb95-65ed-465b-b2b2-b4331f2de544 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:41.358084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:41.358677 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:41.359011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:41.359316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:41.367427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:41.370221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:41.377074 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1422/2826] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:13:42.027884 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026bb3f-1a70-40d8-85dd-7483d25c0234 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:42.028258 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026bb3f-1a70-40d8-85dd-7483d25c0234 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:42.028531 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026bb3f-1a70-40d8-85dd-7483d25c0234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:42.047415 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6026bb3f-1a70-40d8-85dd-7483d25c0234 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:42.048589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6026bb3f-1a70-40d8-85dd-7483d25c0234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:42.057764 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1405/2827] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:13:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:42.070381 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:42.081071 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d250e0-9708-44e4-a234-599333886ee7 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:42.086581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:42.087163 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:42.087440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:42.087709 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:42.095415 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:42.096981 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:42.104753 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1423/2828] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:13:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:13:42.654098 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac65a3-a10b-4640-83d3-fc3ddfd7376a None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:42.654530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac65a3-a10b-4640-83d3-fc3ddfd7376a None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:42.654747 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac65a3-a10b-4640-83d3-fc3ddfd7376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:42.674378 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-59ac65a3-a10b-4640-83d3-fc3ddfd7376a None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:42.675564 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-59ac65a3-a10b-4640-83d3-fc3ddfd7376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:42.683238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1406/2829] 10.222.0.72 () {58 vars in 967 bytes} [Tue Sep 13 13:13:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:42.695226 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:42.705762 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb61ee-6e52-4f2a-a1be-3074b4a184f9 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:42.710795 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:42.711364 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:42.711641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:42.711912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:42.719621 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:42.721196 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:42.729558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1424/2830] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:13:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:13:44.292229 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56553b23-0a62-4ace-9929-ca9960b46388 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:44.292720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56553b23-0a62-4ace-9929-ca9960b46388 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:44.292892 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56553b23-0a62-4ace-9929-ca9960b46388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:44.315234 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-56553b23-0a62-4ace-9929-ca9960b46388 None None] MFA Rules not processed for user `6218b07ed7864e2589de6eff3fd1b08d`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:44.318697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-56553b23-0a62-4ace-9929-ca9960b46388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:44.326400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1407/2831] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:45.132427 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:45.133931 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:45.147783 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae8ae-6ddc-41a2-a650-ccc8d4e1bc11 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:45.152186 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:45.152807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:45.159118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:45.159118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] PATH_INFO: `/v3/users/bea0e9183c47439f81544c5eb9cf6c77` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:45.159118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea0e9183c47439f81544c5eb9cf6c77)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:45.159118 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:45.168070 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:45.346834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea0e9183c47439f81544c5eb9cf6c77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:45.347406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea0e9183c47439f81544c5eb9cf6c77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:45.355390 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea0e9183c47439f81544c5eb9cf6c77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:45.367490 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea0e9183c47439f81544c5eb9cf6c77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:45.368175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea0e9183c47439f81544c5eb9cf6c77'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:45.475392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1425/2832] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:45 2022] DELETE /identity/v3/users/bea0e9183c47439f81544c5eb9cf6c77 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:45.479619 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:45.480085 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:45.480272 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:45.498690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:45.499891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:45.558455 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1408/2833] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:45.569402 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:45.570868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:45.629537 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf7ba6-39e0-40f5-8fa0-3853c67ff315 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:45.633631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0007dbc-acdf-47a0-bbed-386a32b5fb54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:45.639277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0007dbc-acdf-47a0-bbed-386a32b5fb54 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:45.639277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0007dbc-acdf-47a0-bbed-386a32b5fb54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:45.639277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0007dbc-acdf-47a0-bbed-386a32b5fb54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:45.644358 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0007dbc-acdf-47a0-bbed-386a32b5fb54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:45.646041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0007dbc-acdf-47a0-bbed-386a32b5fb54 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:45.656243 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1426/2834] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:46.042615 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:46.044150 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:46.057312 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d033-fcca-4444-8d25-d704fbe2618e None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:46.062653 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:46.063350 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.063598 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.066868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] PATH_INFO: `/v3/projects/d63cdb20ba4b46af98b88cb9867fd275` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.066868 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63cdb20ba4b46af98b88cb9867fd275)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:46.067071 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:46.100850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:46.173851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63cdb20ba4b46af98b88cb9867fd275', 'request_id': 'req-b0a2f76e-b084-498d-9226-2546123d39a7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:46.234630 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63cdb20ba4b46af98b88cb9867fd275', 'request_id': 'req-b0a2f76e-b084-498d-9226-2546123d39a7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:46.242086 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63cdb20ba4b46af98b88cb9867fd275', 'request_id': 'req-b0a2f76e-b084-498d-9226-2546123d39a7'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:46.252744 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] The token cache is being invalidate because project d63cdb20ba4b46af98b88cb9867fd275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:46.253018 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:46.253832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-b0a2f76e-b084-498d-9226-2546123d39a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:46.255344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1409/2835] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:46 2022] DELETE /identity/v3/projects/d63cdb20ba4b46af98b88cb9867fd275 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:46.261379 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176d9b6-edde-4aa6-9d1d-fee775f28110 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.261863 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176d9b6-edde-4aa6-9d1d-fee775f28110 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.261983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176d9b6-edde-4aa6-9d1d-fee775f28110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.282295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-5176d9b6-edde-4aa6-9d1d-fee775f28110 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:46.283558 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-5176d9b6-edde-4aa6-9d1d-fee775f28110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:46.342033 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1427/2836] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:46.350464 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c44de-8a2f-4af9-a2c0-6e7a78c04605 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.350741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c44de-8a2f-4af9-a2c0-6e7a78c04605 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.351009 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c44de-8a2f-4af9-a2c0-6e7a78c04605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.371957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-048c44de-8a2f-4af9-a2c0-6e7a78c04605 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:46.373144 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-048c44de-8a2f-4af9-a2c0-6e7a78c04605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:46.381263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1410/2837] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:46.385596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176d9b6-edde-4aa6-9d1d-fee775f28110 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:46.396803 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176d9b6-edde-4aa6-9d1d-fee775f28110 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:46.400682 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:46.401280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.401559 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.401868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.402510 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:46.402814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:46.404380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:46.416531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1428/2838] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:13:46.420827 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c44de-8a2f-4af9-a2c0-6e7a78c04605 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:46.430830 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c44de-8a2f-4af9-a2c0-6e7a78c04605 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:46.435558 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:46.436145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.436419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.436851 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.437595 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:46.439502 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:46.548373 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1411/2839] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:46 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:13:46.553291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:46.563057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8f1a1-cb1b-4810-a9ad-70d3b0198b14 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:46.575098 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:46.619251 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:46.619853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.620153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.620417 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.621040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:46.622965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:46.698489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1429/2840] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:46 2022] POST /identity/v3/users => generated 300 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:46.702799 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:46.714022 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64e2e2-3a17-45b1-9ce6-c5edcc291817 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:46.718807 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:46.719463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.719743 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.720016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.720664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:46.722292 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:46.732788 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1412/2841] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:46.736742 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:46.746765 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101cb45-fa69-4ccd-92e2-d8b18d1bd802 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:46.751362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc14ea-bc9a-4d1c-82ad-d4f0d79796dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:46.751952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc14ea-bc9a-4d1c-82ad-d4f0d79796dc admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.752228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc14ea-bc9a-4d1c-82ad-d4f0d79796dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.752502 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc14ea-bc9a-4d1c-82ad-d4f0d79796dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.753139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc14ea-bc9a-4d1c-82ad-d4f0d79796dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:46.754728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc14ea-bc9a-4d1c-82ad-d4f0d79796dc admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:46.767137 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1430/2842] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:13:46.772364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:46.783935 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed10dd-4216-4865-897f-efb71338c6b2 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:46.789838 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:46.790610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.790903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.791178 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] PATH_INFO: `/v3/projects/b851820b983f402a8c3ed75883365cd9/users/76c923be99264b9d92c9e1bb3849338f/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.806931 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b851820b983f402a8c3ed75883365cd9, user_id=76c923be99264b9d92c9e1bb3849338f, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:46.809640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:46.909663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1413/2843] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:46 2022] PUT /identity/v3/projects/b851820b983f402a8c3ed75883365cd9/users/76c923be99264b9d92c9e1bb3849338f/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:46.920851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:46.921126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:46.921398 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:46.949724 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] MFA Rules not processed for user `76c923be99264b9d92c9e1bb3849338f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:46.950965 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:47.013879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1431/2844] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:47.025681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:47.027398 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:47.093968 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ced6-1f20-478f-a622-d157ebaf32db admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:47.098494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45750a20-5757-4c48-a89d-f5293fbd0c1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c5b4fdb65514ca6b8033f0f1e4c5eab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:47.099496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45750a20-5757-4c48-a89d-f5293fbd0c1c service cinder] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:47.099781 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45750a20-5757-4c48-a89d-f5293fbd0c1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:47.100052 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45750a20-5757-4c48-a89d-f5293fbd0c1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:47.109012 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45750a20-5757-4c48-a89d-f5293fbd0c1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:47.110501 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45750a20-5757-4c48-a89d-f5293fbd0c1c service cinder] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:47.120500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1414/2845] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:47 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:47.893994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:47.895435 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:47.965011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648718b-eb1a-4d7a-9bdf-ef6f2e2dcf2d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:47.969851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:47.970505 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:47.970913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:47.971199 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] PATH_INFO: `/v3/users/833622fc7358412ba4c0c092ee8bbb61` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:47.972467 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=833622fc7358412ba4c0c092ee8bbb61)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:47.974422 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:48.058936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:48.173549 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833622fc7358412ba4c0c092ee8bbb61'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:48.174316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833622fc7358412ba4c0c092ee8bbb61'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:48.182057 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833622fc7358412ba4c0c092ee8bbb61'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:48.194269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833622fc7358412ba4c0c092ee8bbb61'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:48.200192 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833622fc7358412ba4c0c092ee8bbb61'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:48.281649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1432/2846] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:47 2022] DELETE /identity/v3/users/833622fc7358412ba4c0c092ee8bbb61 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:48.283673 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:48.284164 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:48.284263 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:48.313610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:48.314966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:48.381924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:48.405249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1415/2847] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:48.414933 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:48.417277 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:48.479360 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fefd84-c0da-4135-8758-711a68e6fc8a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:48.483838 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7ed75-9148-42ee-a6d1-47f04ddc4529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:48.484440 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7ed75-9148-42ee-a6d1-47f04ddc4529 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:48.484715 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7ed75-9148-42ee-a6d1-47f04ddc4529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:48.484989 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7ed75-9148-42ee-a6d1-47f04ddc4529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:48.494956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7ed75-9148-42ee-a6d1-47f04ddc4529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:48.496408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7ed75-9148-42ee-a6d1-47f04ddc4529 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:48.506591 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1433/2848] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:48.794589 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:48.796681 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:48.810050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255cbbe5-00a2-4985-8300-5dedccd16c45 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:48.815786 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:48.816463 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:48.816740 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:48.817208 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] PATH_INFO: `/v3/projects/a26d0a86c9d54012b058c7127cc71ecd` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:48.818559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26d0a86c9d54012b058c7127cc71ecd)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:48.821280 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:48.943663 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26d0a86c9d54012b058c7127cc71ecd', 'request_id': 'req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:49.041037 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26d0a86c9d54012b058c7127cc71ecd', 'request_id': 'req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:49.052181 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26d0a86c9d54012b058c7127cc71ecd', 'request_id': 'req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:49.078230 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] The token cache is being invalidate because project a26d0a86c9d54012b058c7127cc71ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:49.078847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:49.080077 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-f7a7a691-bfc6-42bb-8c3e-61fa29bcd74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:49.082662 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1416/2849] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:48 2022] DELETE /identity/v3/projects/a26d0a86c9d54012b058c7127cc71ecd => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:49.087409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce423168-023b-456f-9fd0-79182358997f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.088326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce423168-023b-456f-9fd0-79182358997f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.088681 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce423168-023b-456f-9fd0-79182358997f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.127065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-ce423168-023b-456f-9fd0-79182358997f None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:49.128355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-ce423168-023b-456f-9fd0-79182358997f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:49.200229 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1434/2850] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:49.208733 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103090-a417-470c-ae5e-e56940a9e04d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.209023 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103090-a417-470c-ae5e-e56940a9e04d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.209307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103090-a417-470c-ae5e-e56940a9e04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.232100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-aa103090-a417-470c-ae5e-e56940a9e04d None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:49.233316 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-aa103090-a417-470c-ae5e-e56940a9e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:49.242876 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1417/2851] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:49.247531 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce423168-023b-456f-9fd0-79182358997f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:49.258613 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce423168-023b-456f-9fd0-79182358997f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:49.263636 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:49.264388 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.264672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.264951 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.265657 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:49.266005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:49.267668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:49.281797 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1435/2852] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:13:49.286447 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103090-a417-470c-ae5e-e56940a9e04d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:49.298667 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103090-a417-470c-ae5e-e56940a9e04d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:49.309004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:49.309836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.310215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.310776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.311521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:49.313891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:49.426135 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1418/2853] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:49 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:49.426755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:49.444222 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504687c0-2544-4b8a-b0c5-b1ebcfe3b2ab admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:49.488285 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:49.515319 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:49.516891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.517312 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.517717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.518491 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:49.520617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:49.655515 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1436/2854] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:49 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:49.661054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:49.674833 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c72313-b2eb-459c-87f4-b55fad8b9e0e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:49.682191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:49.682191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.682191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.682191 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.682795 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:49.684074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:49.696338 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1419/2855] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:13:49.703935 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:49.714453 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ca3cc-f925-4dcf-aa54-e2bca635c0ea admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:49.719321 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593c4c0-a4d4-4042-b4e1-83c2fab8e000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:49.720019 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593c4c0-a4d4-4042-b4e1-83c2fab8e000 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.720353 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593c4c0-a4d4-4042-b4e1-83c2fab8e000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.720580 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593c4c0-a4d4-4042-b4e1-83c2fab8e000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.721227 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593c4c0-a4d4-4042-b4e1-83c2fab8e000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:49.722917 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593c4c0-a4d4-4042-b4e1-83c2fab8e000 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:49.733472 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1437/2856] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:13:49.737849 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:49.747720 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aad516-6aba-431a-a805-21b6a163afaa admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:49.752982 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:49.753640 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.757435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.757435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] PATH_INFO: `/v3/projects/aa5bc3bf3bb74d6a91e79dd861d0563f/users/ede50e164bea4a89a28e50243d145447/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.769126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5bc3bf3bb74d6a91e79dd861d0563f, user_id=ede50e164bea4a89a28e50243d145447, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:49.771796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:49.887048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1420/2857] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:49 2022] PUT /identity/v3/projects/aa5bc3bf3bb74d6a91e79dd861d0563f/users/ede50e164bea4a89a28e50243d145447/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:49.892206 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbaecb-3172-45da-a1ca-ecc5389bd2eb None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:49.892494 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbaecb-3172-45da-a1ca-ecc5389bd2eb None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:49.892769 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbaecb-3172-45da-a1ca-ecc5389bd2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:49.912907 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-90cbaecb-3172-45da-a1ca-ecc5389bd2eb None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:49.917976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-90cbaecb-3172-45da-a1ca-ecc5389bd2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:49.979094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1438/2858] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:49.985602 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:49.987364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:50.046356 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbe2bd-2d66-4794-b337-1c9a85ae9dbc admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:50.050025 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:50.050646 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:50.050919 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:50.051193 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:50.061771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:50.063592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:50.073105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1421/2859] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:55.418507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a152c-cd4f-40b3-bcb0-68e0ec916a97 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:55.419035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a152c-cd4f-40b3-bcb0-68e0ec916a97 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:55.419035 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a152c-cd4f-40b3-bcb0-68e0ec916a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:55.453082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-188a152c-cd4f-40b3-bcb0-68e0ec916a97 None None] MFA Rules not processed for user `ede50e164bea4a89a28e50243d145447`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:55.454378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-188a152c-cd4f-40b3-bcb0-68e0ec916a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:55.556351 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1439/2860] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:55.563500 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:55.574142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c555f-5ca5-48a2-87d3-cea23d34b456 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:55.580678 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:55.581294 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:55.581774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:55.582307 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] PATH_INFO: `/v3/projects` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:55.583115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:55.585358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:55.602986 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:55.719690 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1422/2861] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:55 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:55.723755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a152c-cd4f-40b3-bcb0-68e0ec916a97 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:55.735465 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a152c-cd4f-40b3-bcb0-68e0ec916a97 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:55.791458 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:55.807647 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:55.808581 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:55.809027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:55.809306 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:55.810063 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:55.811914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:56.025334 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1440/2862] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:55 2022] POST /identity/v3/users => generated 298 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:56.029399 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:56.043576 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a6d4d-71f2-4117-aa5d-d216e2a73834 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:56.051237 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:56.051854 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:56.052180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:56.053572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:56.053572 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:56.055336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:56.069582 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1423/2863] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:56.070932 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:56.082040 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fe51a-693e-443d-b1c9-ee9f5e21fce9 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:56.087182 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:56.089879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:56.089879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:56.089879 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] PATH_INFO: `/v3/projects/6208cd035241428eb8f820b1088bcf90/users/e5e02f485f934a5f97c368cb8b76dc5c/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:56.106430 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6208cd035241428eb8f820b1088bcf90, user_id=e5e02f485f934a5f97c368cb8b76dc5c, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:56.109161 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:56.189155 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1441/2864] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:56 2022] PUT /identity/v3/projects/6208cd035241428eb8f820b1088bcf90/users/e5e02f485f934a5f97c368cb8b76dc5c/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:56.203145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:56.208075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f9392-ba5e-40db-9d4c-9b2915da6325 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:56.271304 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:56.271912 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:56.272196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:56.272541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:56.273261 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:56.274934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:56.285225 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1424/2865] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:56.292809 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:56.301450 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4399012-f6b4-470d-ac5b-830f4e17c43a admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:56.307733 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2cfec-fe26-4697-85a2-48f3f324ab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:56.308474 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2cfec-fe26-4697-85a2-48f3f324ab29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:56.308757 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2cfec-fe26-4697-85a2-48f3f324ab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:56.309034 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2cfec-fe26-4697-85a2-48f3f324ab29 admin admin] PATH_INFO: `/v3/projects/6208cd035241428eb8f820b1088bcf90/users/e5e02f485f934a5f97c368cb8b76dc5c/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:56.311041 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2cfec-fe26-4697-85a2-48f3f324ab29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6208cd035241428eb8f820b1088bcf90, user_id=e5e02f485f934a5f97c368cb8b76dc5c, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:56.313860 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2cfec-fe26-4697-85a2-48f3f324ab29 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:56.324997 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1442/2866] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:56 2022] PUT /identity/v3/projects/6208cd035241428eb8f820b1088bcf90/users/e5e02f485f934a5f97c368cb8b76dc5c/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:56.411611 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:56.414005 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:56.426746 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd31ec-fd3a-420a-b9ba-7e4897d3bdb1 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:56.490664 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:56.491551 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:56.491882 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:56.492196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] PATH_INFO: `/v3/users/76c923be99264b9d92c9e1bb3849338f` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:56.493526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c923be99264b9d92c9e1bb3849338f)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:56.495329 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:56.742741 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c923be99264b9d92c9e1bb3849338f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:56.743891 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c923be99264b9d92c9e1bb3849338f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:56.752914 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c923be99264b9d92c9e1bb3849338f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:56.767800 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c923be99264b9d92c9e1bb3849338f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:56.769054 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c923be99264b9d92c9e1bb3849338f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:56.773967 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:56.875736 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1425/2867] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:13:56 2022] DELETE /identity/v3/users/76c923be99264b9d92c9e1bb3849338f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:56.880342 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:56.880853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:56.880853 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:56.915297 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:56.916836 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:57.004050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1443/2868] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:57.034783 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:57.036256 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:57.104131 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebcd29-f87f-43d4-88e1-0f020dce7a5a admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:57.107824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc1268-2e6c-42c2-a8b6-7787024dc4b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:57.108431 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc1268-2e6c-42c2-a8b6-7787024dc4b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:57.108713 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc1268-2e6c-42c2-a8b6-7787024dc4b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:57.108984 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc1268-2e6c-42c2-a8b6-7787024dc4b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:57.119108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc1268-2e6c-42c2-a8b6-7787024dc4b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:57.120559 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc1268-2e6c-42c2-a8b6-7787024dc4b3 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:57.131313 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1426/2869] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:13:57.676436 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:57.678130 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:57.695538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6cdb6-d7db-4543-828a-7e659fe21cbd None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:57.701065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:57.701786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:57.703261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:57.703261 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] PATH_INFO: `/v3/projects/b851820b983f402a8c3ed75883365cd9` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:57.703683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b851820b983f402a8c3ed75883365cd9)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:57.705856 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:57.861441 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b851820b983f402a8c3ed75883365cd9', 'request_id': 'req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:57.925632 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b851820b983f402a8c3ed75883365cd9', 'request_id': 'req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:57.934787 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b851820b983f402a8c3ed75883365cd9', 'request_id': 'req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:57.951405 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] The token cache is being invalidate because project b851820b983f402a8c3ed75883365cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:13:57.952094 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:57.952976 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-9e5dda87-fbd4-4ae7-9b97-ef211e26a9ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:13:57.961190 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea50db-806f-481a-9c93-270e4c661993 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:57.962119 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea50db-806f-481a-9c93-270e4c661993 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:57.962499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea50db-806f-481a-9c93-270e4c661993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:57.965517 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1444/2870] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:13:57 2022] DELETE /identity/v3/projects/b851820b983f402a8c3ed75883365cd9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:13:57.985278 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-6aea50db-806f-481a-9c93-270e4c661993 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:57.987654 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6aea50db-806f-481a-9c93-270e4c661993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:58.072521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1427/2871] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:58.081894 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0212-e033-44ac-a27b-5ab69fc9f0a2 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.088763 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0212-e033-44ac-a27b-5ab69fc9f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.089706 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0212-e033-44ac-a27b-5ab69fc9f0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.112082 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-962f0212-e033-44ac-a27b-5ab69fc9f0a2 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:58.113310 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-962f0212-e033-44ac-a27b-5ab69fc9f0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:58.122125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1445/2872] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:58.126239 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea50db-806f-481a-9c93-270e4c661993 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:58.144808 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea50db-806f-481a-9c93-270e4c661993 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:58.150803 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:58.154348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.154348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.154348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.154348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81578) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:13:58.154348 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:58.155147 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:58.173311 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1428/2873] 10.222.0.72 () {58 vars in 1153 bytes} [Tue Sep 13 13:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:13:58.178060 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f0212-e033-44ac-a27b-5ab69fc9f0a2 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:58.191947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f0212-e033-44ac-a27b-5ab69fc9f0a2 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:58.198767 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:58.200240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.200240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.200240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.200240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:58.202740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:58.217828 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:58.311897 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:58.318758 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1446/2874] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:13:58 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:58.323584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cc2f9-8dc8-4de2-9472-5ca6dba303c9 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:58.410883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:58.411703 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.412162 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.412622 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.414013 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:58.416238 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:58.589774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1429/2875] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:13:58 2022] POST /identity/v3/users => generated 290 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:13:58.590000 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:58.595507 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3e94d-43dc-43dd-8582-4e68c2c29d71 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:58.606025 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:58.607089 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.607565 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.607996 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.608818 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:58.610694 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:58.624463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1447/2876] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:13:58.628591 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:58.643904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992cedd-1f0a-4d34-b0d1-ee58d76d2eca admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:58.652377 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5156fb-afb2-4f7c-aa36-515fdc4c663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:58.653243 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5156fb-afb2-4f7c-aa36-515fdc4c663e admin admin] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.664064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5156fb-afb2-4f7c-aa36-515fdc4c663e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.664064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5156fb-afb2-4f7c-aa36-515fdc4c663e admin admin] PATH_INFO: `/v3/roles` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.664064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5156fb-afb2-4f7c-aa36-515fdc4c663e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:58.664064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5156fb-afb2-4f7c-aa36-515fdc4c663e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:58.673108 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1430/2877] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 13:13:58.674240 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:58.687268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8ebb-0830-4196-8474-b654c920be76 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:58.692600 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:58.693330 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.698859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.698859 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] PATH_INFO: `/v3/projects/bdc958fc46ca40afa50f4e7f43a0ffe1/users/44492f86f0cb4d4ea1fee7ea2ae7d10f/roles/4279c61c3b4245e39915c7d4d0293dfe` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.714457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc958fc46ca40afa50f4e7f43a0ffe1, user_id=44492f86f0cb4d4ea1fee7ea2ae7d10f, role_id=4279c61c3b4245e39915c7d4d0293dfe)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:58.717755 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:58.801078 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1448/2878] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:13:58 2022] PUT /identity/v3/projects/bdc958fc46ca40afa50f4e7f43a0ffe1/users/44492f86f0cb4d4ea1fee7ea2ae7d10f/roles/4279c61c3b4245e39915c7d4d0293dfe => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:13:58.810627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c03f7-a639-4b8b-95dc-9be6b41dcb43 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:58.810627 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c03f7-a639-4b8b-95dc-9be6b41dcb43 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:58.810889 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c03f7-a639-4b8b-95dc-9be6b41dcb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:58.823633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:13:58.841142 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-766c03f7-a639-4b8b-95dc-9be6b41dcb43 None None] MFA Rules not processed for user `259c064f989f4405a7a84f397fec6223`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:13:58.844271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-766c03f7-a639-4b8b-95dc-9be6b41dcb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:58.980055 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1431/2879] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:13:58.992834 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:13:58.994941 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:13:59.062807 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d33dcd-c4c1-44a3-bd0e-62edd1706ca0 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:13:59.066655 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:13:59.067348 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:13:59.067930 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:13:59.068395 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:13:59.078988 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:13:59.080425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:13:59.091526 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1449/2880] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:14:04.127848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148a00b-d87d-465b-ae56-a14767d7762c None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:04.128413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148a00b-d87d-465b-ae56-a14767d7762c None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:04.128413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148a00b-d87d-465b-ae56-a14767d7762c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:04.167136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f148a00b-d87d-465b-ae56-a14767d7762c None None] MFA Rules not processed for user `e5e02f485f934a5f97c368cb8b76dc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:04.168796 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f148a00b-d87d-465b-ae56-a14767d7762c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:04.296496 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1432/2881] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:14:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:04.315258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:04.315258 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:04.393305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cef78-66b6-4dea-89fa-c0d8586a413c service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:04.404172 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04084ffe1b0548fea75ea144977e4f15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:04.405963 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:04.406256 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:04.406539 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:04.409030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:04.537362 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:04.539599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:04.550820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1450/2882] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:14:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:14:04.982658 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e4928-9e52-4961-9a0d-6bdd0839dfce None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:04.985873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e4928-9e52-4961-9a0d-6bdd0839dfce None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:04.985873 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e4928-9e52-4961-9a0d-6bdd0839dfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:05.006140 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-938e4928-9e52-4961-9a0d-6bdd0839dfce None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:05.008554 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-938e4928-9e52-4961-9a0d-6bdd0839dfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:05.097768 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1433/2883] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:14:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:05.116536 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:05.118475 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:05.139713 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:05.219018 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427317a-b978-441b-90db-80c6431f8a7c service glance] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:05.224577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:05.225289 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:05.225596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:05.225911 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:05.235268 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:05.236905 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:05.252805 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1451/2884] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:14:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:14:07.477798 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96133d5-f524-4208-b347-0563f5a2e8ee None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:07.478665 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96133d5-f524-4208-b347-0563f5a2e8ee None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:07.479105 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96133d5-f524-4208-b347-0563f5a2e8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:07.539432 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e96133d5-f524-4208-b347-0563f5a2e8ee None None] MFA Rules not processed for user `44492f86f0cb4d4ea1fee7ea2ae7d10f`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:07.541568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e96133d5-f524-4208-b347-0563f5a2e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:07.616584 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:07.706774 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1434/2885] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:14:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:07.708111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:07.722469 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0861184-9d73-40d1-ba5f-e6ba46d651e6 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:07.733050 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:07.733683 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:07.733994 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:07.763668 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:07.764748 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:07.774744 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:07.909298 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1452/2886] 10.222.0.72 () {60 vars in 1153 bytes} [Tue Sep 13 13:14:07 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:14:07.913505 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96133d5-f524-4208-b347-0563f5a2e8ee None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:07.943529 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96133d5-f524-4208-b347-0563f5a2e8ee None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:08.210771 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:08.211672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:08.212048 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:08.212435 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:08.213207 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:08.215618 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:08.326486 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1435/2887] 10.222.0.72 () {60 vars in 1144 bytes} [Tue Sep 13 13:14:07 2022] POST /identity/v3/users => generated 296 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:14:08.328087 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:08.352489 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8e66-dfde-440d-9753-49d322434daf admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:08.361009 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:08.361649 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:08.361962 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:08.362295 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:08.362956 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:08.365416 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:08.385824 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1453/2888] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:14:08.387544 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:08.404530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f98afb-78a7-46b8-9462-8339230379c4 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:08.418449 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:08.419285 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:08.419669 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:08.420035 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] PATH_INFO: `/v3/projects/fe62e2f040a246f4b60c8145cf9ce48a/users/d4a9c9f4ce0046598cfb950197c963c3/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:08.457577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe62e2f040a246f4b60c8145cf9ce48a, user_id=d4a9c9f4ce0046598cfb950197c963c3, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:08.462903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:08.579843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1436/2889] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:14:08 2022] PUT /identity/v3/projects/fe62e2f040a246f4b60c8145cf9ce48a/users/d4a9c9f4ce0046598cfb950197c963c3/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:14:08.581928 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:08.594287 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbc65-fe64-4303-9627-7e069508f2e5 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:08.750111 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc56e65-3a4b-4be2-bdf1-6fe8edb30337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:08.750925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc56e65-3a4b-4be2-bdf1-6fe8edb30337 admin admin] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:08.751378 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc56e65-3a4b-4be2-bdf1-6fe8edb30337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:08.753653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc56e65-3a4b-4be2-bdf1-6fe8edb30337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:08.753653 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc56e65-3a4b-4be2-bdf1-6fe8edb30337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:08.754538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc56e65-3a4b-4be2-bdf1-6fe8edb30337 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:08.766481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1454/2890] 10.222.0.72 () {58 vars in 1122 bytes} [Tue Sep 13 13:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 13:14:08.770599 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:08.782440 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c5c05-7d20-425f-9eea-fdc4fa5df15e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:08.787883 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148d28f-68b6-4091-ba03-cb5f82a5ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:08.788753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148d28f-68b6-4091-ba03-cb5f82a5ba99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:08.789028 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148d28f-68b6-4091-ba03-cb5f82a5ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:08.789326 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148d28f-68b6-4091-ba03-cb5f82a5ba99 admin admin] PATH_INFO: `/v3/projects/fe62e2f040a246f4b60c8145cf9ce48a/users/d4a9c9f4ce0046598cfb950197c963c3/roles/0c2f57ac71af4bb6be296c0ff7db64de` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:08.791442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148d28f-68b6-4091-ba03-cb5f82a5ba99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe62e2f040a246f4b60c8145cf9ce48a, user_id=d4a9c9f4ce0046598cfb950197c963c3, role_id=0c2f57ac71af4bb6be296c0ff7db64de)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:08.794575 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148d28f-68b6-4091-ba03-cb5f82a5ba99 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:08.808324 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1437/2891] 10.222.0.72 () {60 vars in 1483 bytes} [Tue Sep 13 13:14:08 2022] PUT /identity/v3/projects/fe62e2f040a246f4b60c8145cf9ce48a/users/d4a9c9f4ce0046598cfb950197c963c3/roles/0c2f57ac71af4bb6be296c0ff7db64de => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:14:10.980561 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775983f4-e256-490d-aa07-d84813a12457 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:10.981535 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775983f4-e256-490d-aa07-d84813a12457 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:10.982027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775983f4-e256-490d-aa07-d84813a12457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:11.004672 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-775983f4-e256-490d-aa07-d84813a12457 None None] MFA Rules not processed for user `6218b07ed7864e2589de6eff3fd1b08d`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:11.006026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-775983f4-e256-490d-aa07-d84813a12457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:11.119027 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1455/2892] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:14:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:17.597347 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6186a-e883-435c-ab36-7eb1c0f3bc88 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:17.597902 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6186a-e883-435c-ab36-7eb1c0f3bc88 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:17.597991 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6186a-e883-435c-ab36-7eb1c0f3bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:17.677308 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-bbe6186a-e883-435c-ab36-7eb1c0f3bc88 None None] MFA Rules not processed for user `d4a9c9f4ce0046598cfb950197c963c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:17.680064 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-bbe6186a-e883-435c-ab36-7eb1c0f3bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:17.811295 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1438/2893] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:14:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:17.825814 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775983f4-e256-490d-aa07-d84813a12457 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:17.827377 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-775983f4-e256-490d-aa07-d84813a12457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:17.844097 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:17.984157 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775983f4-e256-490d-aa07-d84813a12457 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:17.995666 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e24c84-70c6-4b33-b9e4-9d630b4f2c30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:17.996624 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e24c84-70c6-4b33-b9e4-9d630b4f2c30 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:17.997084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e24c84-70c6-4b33-b9e4-9d630b4f2c30 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:17.997851 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e24c84-70c6-4b33-b9e4-9d630b4f2c30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:18.157936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e24c84-70c6-4b33-b9e4-9d630b4f2c30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:18.157936 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e24c84-70c6-4b33-b9e4-9d630b4f2c30 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:18.168870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1456/2894] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:14:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:14:22.764315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca29be-8f82-4e11-9a78-b5647c265247 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:22.764792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca29be-8f82-4e11-9a78-b5647c265247 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:22.764894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca29be-8f82-4e11-9a78-b5647c265247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:22.773315 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:22.786019 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-e2ca29be-8f82-4e11-9a78-b5647c265247 None None] MFA Rules not processed for user `ede50e164bea4a89a28e50243d145447`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:22.787957 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-e2ca29be-8f82-4e11-9a78-b5647c265247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:22.848248 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1439/2895] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:14:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:26.708481 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:26.709522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:26.709625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:26.734030 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:26.735744 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:26.794577 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1457/2896] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:14:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:26.807216 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca29be-8f82-4e11-9a78-b5647c265247 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:26.881339 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca29be-8f82-4e11-9a78-b5647c265247 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:26.886697 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:26.887427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:26.887593 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:26.887848 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:26.896465 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:26.898245 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:26.907472 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1440/2897] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:14:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:14:29.715634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:29.720631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:29.789176 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:29.809275 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61055b5c-6724-4126-8d1c-e5b620618a5d None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:29.813914 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:29.814626 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:29.814904 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:29.815173 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] PATH_INFO: `/v3/users/81634611141f4909a52ac46d5164f17f` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:29.816308 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81634611141f4909a52ac46d5164f17f)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:29.818290 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:30.026792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81634611141f4909a52ac46d5164f17f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.027466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81634611141f4909a52ac46d5164f17f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.036181 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81634611141f4909a52ac46d5164f17f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.050641 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81634611141f4909a52ac46d5164f17f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.051454 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-42e0f7f3-aa16-448f-be0b-ce8ea8adb5f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81634611141f4909a52ac46d5164f17f'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.124813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1458/2898] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:14:29 2022] DELETE /identity/v3/users/81634611141f4909a52ac46d5164f17f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:14:30.652232 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:30.673997 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578335fd-0606-4461-9717-6835fbb74cd8 service swift] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:30.763657 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:30.764291 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:30.764570 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:30.764843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] PATH_INFO: `/v3/projects/15bcfc71833149c194401e5afb6e5d18` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:30.765994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15bcfc71833149c194401e5afb6e5d18)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:30.767869 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:30.875271 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:30.883426 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15bcfc71833149c194401e5afb6e5d18', 'request_id': 'req-c623ed51-2962-4516-8f32-4cf56d7fa9e0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.959536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15bcfc71833149c194401e5afb6e5d18', 'request_id': 'req-c623ed51-2962-4516-8f32-4cf56d7fa9e0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.967046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15bcfc71833149c194401e5afb6e5d18', 'request_id': 'req-c623ed51-2962-4516-8f32-4cf56d7fa9e0'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.981318 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] The token cache is being invalidate because project 15bcfc71833149c194401e5afb6e5d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:14:30.981623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.982293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-c623ed51-2962-4516-8f32-4cf56d7fa9e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:14:30.985247 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1441/2899] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:14:30 2022] DELETE /identity/v3/projects/15bcfc71833149c194401e5afb6e5d18 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:14:39.379011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:39.379463 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:39.379721 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:39.400408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] MFA Rules not processed for user `44492f86f0cb4d4ea1fee7ea2ae7d10f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:39.401634 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:39.458868 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1459/2900] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:14:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:40.977927 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:48.207180 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:48.207742 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:48.208346 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:48.232169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:14:48.232169 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:48.294536 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1442/2901] 10.222.0.72 () {58 vars in 1009 bytes} [Tue Sep 13 13:14:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:14:48.325264 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:48.327188 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:48.412283 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b5a7d-336c-4c2f-8713-beff2e3c9aa5 None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:48.417397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:48.418029 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:48.418313 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:48.418583 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:48.427083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:48.429201 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:48.438099 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1460/2902] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:14:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 13:14:56.238843 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:56.241065 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:56.312488 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c15913-0304-48b7-aa0c-5482fc38d70d None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:56.317975 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:56.318281 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:56.318989 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:56.319128 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:56.321410 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:56.440712 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:56.442706 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:56.463719 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1443/2903] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:14:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:14:56.889579 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:14:56.891519 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:56.906177 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e73ce9-b5c1-403d-a23e-4cb127ff5131 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:14:56.910235 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:14:56.910855 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:14:56.911125 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:14:56.911400 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:14:56.914406 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:14:56.921083 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:14:57.014421 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:14:57.016141 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:14:57.027468 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1461/2904] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:14:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:15:01.555427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:15:01.570890 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dc34f-b895-4c70-a80d-326b4ba66299 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:15:01.577419 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901d864-b2b9-487b-8f02-9e8974e996ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:15:01.579184 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901d864-b2b9-487b-8f02-9e8974e996ac service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:15:01.579904 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901d864-b2b9-487b-8f02-9e8974e996ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:15:01.580492 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901d864-b2b9-487b-8f02-9e8974e996ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:15:01.583001 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-5901d864-b2b9-487b-8f02-9e8974e996ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:15:01.605903 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901d864-b2b9-487b-8f02-9e8974e996ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:15:01.609992 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901d864-b2b9-487b-8f02-9e8974e996ac service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:15:01.637966 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1444/2905] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:15:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:15:02.113973 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:15:02.124190 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e262a5-39a9-4f7b-a8ef-6c0e62340c46 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:15:02.128326 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:15:02.129230 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:15:02.129729 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:15:02.130296 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:15:02.132582 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:15:02.225595 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:15:02.227445 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:15:02.240504 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1462/2906] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:15:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:15:07.036762 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:15:07.037392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:15:07.037392 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:15:07.056726 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:15:07.057951 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:15:07.064976 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1445/2907] 10.222.0.72 () {58 vars in 967 bytes} [Tue Sep 13 13:15:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:15:07.077494 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:15:07.089017 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4f8ab-38ea-42e0-9992-b6324179af5f service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:15:07.094047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:15:07.094642 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:15:07.094925 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:15:07.095200 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:15:07.103047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:15:07.104599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:15:07.112602 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1463/2908] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:15:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:15:09.924452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:15:09.926366 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:15:09.960075 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:15:09.995314 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba2b18-380a-468d-bed7-45c787266002 None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:15:09.999776 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:15:10.000396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:15:10.000672 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:15:10.000950 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] PATH_INFO: `/v3/users/6218b07ed7864e2589de6eff3fd1b08d` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:15:10.002080 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6218b07ed7864e2589de6eff3fd1b08d)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:15:10.004016 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:15:10.137004 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6218b07ed7864e2589de6eff3fd1b08d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.137594 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6218b07ed7864e2589de6eff3fd1b08d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.144439 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6218b07ed7864e2589de6eff3fd1b08d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.156921 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6218b07ed7864e2589de6eff3fd1b08d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.157586 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6218b07ed7864e2589de6eff3fd1b08d'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.236694 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1446/2909] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:15:09 2022] DELETE /identity/v3/users/6218b07ed7864e2589de6eff3fd1b08d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:15:10.473142 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:15:10.486611 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6e473-422c-40a4-a442-008df80a28f6 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:15:10.546221 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:15:10.546843 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:15:10.547122 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:15:10.547399 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] PATH_INFO: `/v3/projects/a5d396f1d5c14e179ab56809f912f1dc` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:15:10.548493 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d396f1d5c14e179ab56809f912f1dc)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:15:10.550409 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:15:10.575166 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:15:10.688952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d396f1d5c14e179ab56809f912f1dc', 'request_id': 'req-8530159a-2c0a-4aae-bea9-63284dadc23b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.763355 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d396f1d5c14e179ab56809f912f1dc', 'request_id': 'req-8530159a-2c0a-4aae-bea9-63284dadc23b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.770922 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d396f1d5c14e179ab56809f912f1dc', 'request_id': 'req-8530159a-2c0a-4aae-bea9-63284dadc23b'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.781920 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] The token cache is being invalidate because project a5d396f1d5c14e179ab56809f912f1dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:15:10.782214 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.782897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:15:10.784389 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1464/2910] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:15:10 2022] DELETE /identity/v3/projects/a5d396f1d5c14e179ab56809f912f1dc => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:15:20.233130 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:15:20.778785 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:15:27.602026 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:15:27.603620 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:15:27.668994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c53a7b-5e86-434c-839e-a31b9b6d528e admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:15:27.716126 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35341e7e-4e7b-4333-b846-50f2e38e6204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:15:27.716651 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341e7e-4e7b-4333-b846-50f2e38e6204 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:15:27.716926 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341e7e-4e7b-4333-b846-50f2e38e6204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:15:27.717196 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35341e7e-4e7b-4333-b846-50f2e38e6204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:15:27.718917 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-35341e7e-4e7b-4333-b846-50f2e38e6204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:15:27.728841 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35341e7e-4e7b-4333-b846-50f2e38e6204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:15:27.730364 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35341e7e-4e7b-4333-b846-50f2e38e6204 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:15:27.738252 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1447/2911] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:15:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:15:37.734675 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=46 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:16:13.681091 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:16:13.683760 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:13.779324 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530159a-2c0a-4aae-bea9-63284dadc23b admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:16:13.785993 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:16:13.787305 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:13.788934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:13.788934 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:13.791350 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:13.806343 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:16:13.810778 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:16:13.823153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1465/2912] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:16:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:16:22.637039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] REQUEST_METHOD: `POST` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:22.637513 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:22.637623 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:22.689913 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.auth.core [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] MFA Rules not processed for user `e658441f5ed34cfbb926364644fbaed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81578) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:16:22.691163 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:22.755555 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1448/2913] 10.222.0.72 () {58 vars in 967 bytes} [Tue Sep 13 13:16:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:16:22.902232 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:16:22.903651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:22.991913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbc19e-8ba6-4572-9a94-e806057977b3 service nova] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:16:22.997011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a08e9745841d4716a64700da59eebf04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:16:22.997625 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:22.997954 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:22.998228 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:23.006084 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:16:23.008328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:16:23.016145 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1466/2914] 10.222.0.72 () {60 vars in 1384 bytes} [Tue Sep 13 13:16:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:16:27.582610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:16:27.584115 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:27.655790 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c86e9-cdd2-4e02-b9b5-779026c46dec None None] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:16:27.661249 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:16:27.661923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:27.662206 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:27.662481 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:27.664661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:27.677671 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:16:27.679288 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:16:27.689687 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1449/2915] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:16:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:16:31.397692 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:16:31.399316 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:31.459880 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=100 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:16:31.465897 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433652-c63c-4493-9eb8-7ff62f98c356 service swift] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:16:31.470622 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:16:31.471195 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:31.471479 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:31.471756 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] PATH_INFO: `/v3/users/ede50e164bea4a89a28e50243d145447` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:31.472896 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede50e164bea4a89a28e50243d145447)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:16:31.474845 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:16:31.673673 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede50e164bea4a89a28e50243d145447'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:31.674392 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede50e164bea4a89a28e50243d145447'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:31.681754 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede50e164bea4a89a28e50243d145447'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:31.694026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede50e164bea4a89a28e50243d145447'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:31.698516 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede50e164bea4a89a28e50243d145447'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:31.791596 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1467/2916] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:16:31 2022] DELETE /identity/v3/users/ede50e164bea4a89a28e50243d145447 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:16:32.076309 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:16:32.090677 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b714-3ba2-4535-86a1-1970f8083620 service placement] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:16:32.148427 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:16:32.149067 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:32.149341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:32.149641 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] PATH_INFO: `/v3/projects/aa5bc3bf3bb74d6a91e79dd861d0563f` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:32.151053 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5bc3bf3bb74d6a91e79dd861d0563f)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:16:32.152934 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:16:32.155259 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=100 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:16:32.276006 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5bc3bf3bb74d6a91e79dd861d0563f', 'request_id': 'req-21538d89-da1a-404d-8040-8e9626a14885'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.338643 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5bc3bf3bb74d6a91e79dd861d0563f', 'request_id': 'req-21538d89-da1a-404d-8040-8e9626a14885'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.345176 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5bc3bf3bb74d6a91e79dd861d0563f', 'request_id': 'req-21538d89-da1a-404d-8040-8e9626a14885'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.355479 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] The token cache is being invalidate because project aa5bc3bf3bb74d6a91e79dd861d0563f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:16:32.355766 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.356396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.357894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1450/2917] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:16:32 2022] DELETE /identity/v3/projects/aa5bc3bf3bb74d6a91e79dd861d0563f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:16:32.361958 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:16:32.363365 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:32.377026 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928ba16-72d4-47e1-9b2b-1529aa01c746 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:16:32.432408 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:16:32.433065 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:32.433294 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:32.433574 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] PATH_INFO: `/v3/users/e5e02f485f934a5f97c368cb8b76dc5c` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:32.434740 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e02f485f934a5f97c368cb8b76dc5c)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:16:32.436651 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:16:32.632568 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e02f485f934a5f97c368cb8b76dc5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.633291 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e02f485f934a5f97c368cb8b76dc5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.641224 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e02f485f934a5f97c368cb8b76dc5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.653524 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e02f485f934a5f97c368cb8b76dc5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.654271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-1873c911-6ba6-481d-87c2-4a835eac9357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e02f485f934a5f97c368cb8b76dc5c'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:32.656820 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:16:32.795886 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1468/2918] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:16:32 2022] DELETE /identity/v3/users/e5e02f485f934a5f97c368cb8b76dc5c => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:16:33.041850 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:16:33.043293 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:16:33.055954 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21538d89-da1a-404d-8040-8e9626a14885 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:16:33.112634 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:16:33.113297 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:16:33.113526 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:16:33.113832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] PATH_INFO: `/v3/projects/6208cd035241428eb8f820b1088bcf90` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:16:33.114933 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6208cd035241428eb8f820b1088bcf90)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:16:33.117100 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:16:33.231568 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6208cd035241428eb8f820b1088bcf90', 'request_id': 'req-8afd872d-616a-40c9-9fce-fc7b72f89776'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:33.384475 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6208cd035241428eb8f820b1088bcf90', 'request_id': 'req-8afd872d-616a-40c9-9fce-fc7b72f89776'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:33.391442 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6208cd035241428eb8f820b1088bcf90', 'request_id': 'req-8afd872d-616a-40c9-9fce-fc7b72f89776'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:33.402847 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] The token cache is being invalidate because project 6208cd035241428eb8f820b1088bcf90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81578) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:16:33.403133 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:33.403753 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:16:33.405224 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1451/2919] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:16:33 2022] DELETE /identity/v3/projects/6208cd035241428eb8f820b1088bcf90 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:16:42.875777 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:16:43.399610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=74,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:17:07.693081 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:17:07.693617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:17:07.693617 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:17:07.713086 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] MFA Rules not processed for user `44492f86f0cb4d4ea1fee7ea2ae7d10f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:17:07.714328 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:17:07.768480 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1469/2920] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:17:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:17:08.098400 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:17:08.099792 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:17:08.161341 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd872d-616a-40c9-9fce-fc7b72f89776 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:17:08.164911 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:17:08.165512 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:17:08.165824 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:17:08.166099 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:17:08.167856 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:17:08.177557 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:17:08.181839 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:17:08.188580 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1452/2921] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:17:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:17:08.985117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:17:08.986540 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:17:09.056175 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa686f9d-16a9-4a31-a9f1-10c156188cdf None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:17:09.061234 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b05f90-5e77-4518-9ea4-6c57bb450f7c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:17:09.061873 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05f90-5e77-4518-9ea4-6c57bb450f7c service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:17:09.062151 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05f90-5e77-4518-9ea4-6c57bb450f7c service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:17:09.062423 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b05f90-5e77-4518-9ea4-6c57bb450f7c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:17:09.064133 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-b0b05f90-5e77-4518-9ea4-6c57bb450f7c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:17:09.075560 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b05f90-5e77-4518-9ea4-6c57bb450f7c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:17:09.077117 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b05f90-5e77-4518-9ea4-6c57bb450f7c service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:17:09.087280 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1470/2922] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:17:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:17:18.184403 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=26 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:17:19.082696 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=57 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:17:23.198083 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:17:23.199577 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:17:23.270832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964317a-4242-4b99-b2ac-bc84b052ff8b service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:17:23.276452 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:17:23.277068 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:17:23.277342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:17:23.277610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:17:23.279342 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:17:23.291899 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:17:23.293521 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:17:23.306413 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1453/2923] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:17:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:17:33.301050 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=34 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:19:03.592356 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] REQUEST_METHOD: `POST` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:19:03.592867 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:19:03.592983 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:19:03.612072 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.auth.core [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] MFA Rules not processed for user `44492f86f0cb4d4ea1fee7ea2ae7d10f`. Rule list: `[]` (Enabled: `True`). {{(pid=81577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:19:03.613269 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:19:03.620924 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1471/2924] 10.222.0.72 () {58 vars in 958 bytes} [Tue Sep 13 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 13:19:09.884469 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:19:09.897265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f68fcc-5883-4c0d-a404-4a1537f58681 service nova] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:19:09.902865 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:19:09.903470 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:19:09.903748 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:19:09.904434 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:19:09.906355 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:19:09.918005 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:19:09.919046 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:19:09.919425 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:19:09.920633 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:19:09.932146 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1454/2925] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:19:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:19:09.935912 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed70f5-656e-4442-a1ca-8a69c780573f None None] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:19:09.940891 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:19:09.949947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:19:09.949947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:19:09.949947 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:19:09.951790 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:19:09.953403 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:19:09.964102 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1472/2926] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:19:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 13:19:18.595336 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:19:18.596787 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:19:18.613676 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4442db0-a2bb-4028-a880-a0a37ca8f861 service placement] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:19:18.618994 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b21d991d02b4f95a8d0fcec0a334495', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:19:18.619597 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:19:18.619872 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:19:18.620138 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:19:18.622174 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:19:18.632655 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:19:18.635462 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:19:18.645089 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1455/2927] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:19:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:19:19.959011 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=9 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:19:28.641145 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=12 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:20:58.540679 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:20:58.551728 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4c03c-a4bb-4fee-bbeb-63ddbbc335f5 service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:20:58.555792 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:20:58.556411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:20:58.556850 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:20:58.557126 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:20:58.559643 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:20:58.571669 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:20:58.573061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:20:58.581397 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1473/2928] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:20:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:20:58.881268 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:20:58.891614 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85623d67-7159-4f34-a461-304fb43131ed service nova] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:20:58.898484 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:20:58.899074 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:20:58.899358 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:20:58.902215 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:20:58.905784 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:20:58.922832 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:20:58.924401 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:20:58.947380 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1456/2929] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:20:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:21:00.133680 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:21:00.143455 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8021a5-6598-496e-922f-ec53ec031459 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:21:00.147411 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:21:00.147773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:21:00.148047 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:21:00.148315 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:21:00.150069 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:21:00.164147 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:21:00.165538 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:21:00.177276 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1474/2930] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:21:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:21:08.938944 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=6 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:21:10.172262 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=12 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:21:11.916523 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:21:11.918301 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:21:11.929923 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f9468-7d2d-4d6a-954a-28798139ade8 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:21:11.933530 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ed50a15319e4991ae90b0e5271f4c91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:21:11.934344 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:21:11.934460 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:21:11.934707 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:21:11.937396 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:21:12.000453 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:21:12.001924 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:21:12.011273 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1457/2931] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:21:15.674770 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:21:15.686773 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec4b0a-89cf-4063-a868-a1380bcda3a2 service neutron] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:21:15.691870 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:21:15.695736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] REQUEST_METHOD: `GET` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:21:15.695736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:21:15.695736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:21:15.695736 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:21:15.707213 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:21:15.708794 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:21:15.719466 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1475/2932] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:21:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:21:19.485686 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:21:19.488265 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:21:19.500958 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ab3c8-7fe6-4606-b1ae-f3dfb4940b42 service neutron] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:21:19.505835 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:21:19.506494 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:21:19.506773 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:21:19.507070 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] PATH_INFO: `/v3/users/44492f86f0cb4d4ea1fee7ea2ae7d10f` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:21:19.508188 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44492f86f0cb4d4ea1fee7ea2ae7d10f)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:21:19.510136 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:21:19.703051 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44492f86f0cb4d4ea1fee7ea2ae7d10f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:19.703702 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44492f86f0cb4d4ea1fee7ea2ae7d10f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:19.710715 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44492f86f0cb4d4ea1fee7ea2ae7d10f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:19.722417 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44492f86f0cb4d4ea1fee7ea2ae7d10f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:19.723091 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44492f86f0cb4d4ea1fee7ea2ae7d10f'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:19.802862 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1458/2933] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:21:19 2022] DELETE /identity/v3/users/44492f86f0cb4d4ea1fee7ea2ae7d10f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:21:20.027599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:21:20.040717 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753fad-3234-4eea-815e-bbb2a149fd2c service placement] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:21:20.099187 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:21:20.099813 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:21:20.100092 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:21:20.100369 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] PATH_INFO: `/v3/projects/bdc958fc46ca40afa50f4e7f43a0ffe1` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:21:20.101457 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc958fc46ca40afa50f4e7f43a0ffe1)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:21:20.103380 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:21:20.186586 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc958fc46ca40afa50f4e7f43a0ffe1', 'request_id': 'req-10fe1068-445d-4650-b39a-c7d275a8e1b2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.248139 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc958fc46ca40afa50f4e7f43a0ffe1', 'request_id': 'req-10fe1068-445d-4650-b39a-c7d275a8e1b2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.254735 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc958fc46ca40afa50f4e7f43a0ffe1', 'request_id': 'req-10fe1068-445d-4650-b39a-c7d275a8e1b2'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.265233 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] The token cache is being invalidate because project bdc958fc46ca40afa50f4e7f43a0ffe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:21:20.265522 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.266180 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.267913 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1476/2934] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:21:20 2022] DELETE /identity/v3/projects/bdc958fc46ca40afa50f4e7f43a0ffe1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:21:20.272047 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:21:20.273610 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:21:20.287524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eac89-53f8-4228-981a-9896538c0366 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:21:20.344274 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:21:20.344940 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:21:20.345220 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:21:20.345499 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] PATH_INFO: `/v3/users/d4a9c9f4ce0046598cfb950197c963c3` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:21:20.346670 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a9c9f4ce0046598cfb950197c963c3)` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:21:20.348592 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:21:20.354661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:21:20.548661 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a9c9f4ce0046598cfb950197c963c3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.549330 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a9c9f4ce0046598cfb950197c963c3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.556541 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a9c9f4ce0046598cfb950197c963c3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.568818 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a9c9f4ce0046598cfb950197c963c3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.569524 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.notifications [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a9c9f4ce0046598cfb950197c963c3'} {{(pid=81578) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:20.648283 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1459/2935] 10.222.0.72 () {58 vars in 1224 bytes} [Tue Sep 13 13:21:20 2022] DELETE /identity/v3/users/d4a9c9f4ce0046598cfb950197c963c3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:21:20.896738 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Authenticating user token {{(pid=81577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:21:20.898327 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.fernet_utils [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:21:20.912307 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe1068-445d-4650-b39a-c7d275a8e1b2 admin admin] Validating token access rules against request {{(pid=81577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:21:20.969969 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '259c064f989f4405a7a84f397fec6223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdd5b4a37f1d439babc288358ccc8191', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:21:20.970599 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:21:20.970883 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:21:20.971153 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] PATH_INFO: `/v3/projects/fe62e2f040a246f4b60c8145cf9ce48a` {{(pid=81577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:21:20.972271 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe62e2f040a246f4b60c8145cf9ce48a)` {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:21:20.974202 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] RBAC: Authorization granted {{(pid=81577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:21:20.994592 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:21:21.048786 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe62e2f040a246f4b60c8145cf9ce48a', 'request_id': 'req-06dff8c0-dc45-46f0-a009-369390f27326'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:21.111247 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe62e2f040a246f4b60c8145cf9ce48a', 'request_id': 'req-06dff8c0-dc45-46f0-a009-369390f27326'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:21.117906 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe62e2f040a246f4b60c8145cf9ce48a', 'request_id': 'req-06dff8c0-dc45-46f0-a009-369390f27326'} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:21.129631 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] The token cache is being invalidate because project fe62e2f040a246f4b60c8145cf9ce48a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 13:21:21.129952 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:21.130593 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG keystone.notifications [None req-06dff8c0-dc45-46f0-a009-369390f27326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 13:21:21.132061 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: [pid: 81577|app: 0|req: 1477/2936] 10.222.0.72 () {58 vars in 1233 bytes} [Tue Sep 13 13:21:20 2022] DELETE /identity/v3/projects/fe62e2f040a246f4b60c8145cf9ce48a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:21:30.644628 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:21:31.126447 dv-cinder-801354-10-iscsi devstack@keystone.service[81577]: DEBUG dbcounter [-] [81577] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:24:13.923637 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Authenticating user token {{(pid=81578) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:24:13.924097 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:24:14.062202 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11b59e-a115-4280-9f0d-fbcb09d81bd2 admin admin] Validating token access rules against request {{(pid=81578) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:24:14.118894 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9c5e2-88ff-4330-a9d8-946e5231d1a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dca4df9896df47268d091c3b32cbe8d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03397d33d3b1442597a329d1ae8b7543', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81578) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:24:14.119495 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9c5e2-88ff-4330-a9d8-946e5231d1a1 service placement] REQUEST_METHOD: `GET` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:24:14.119772 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9c5e2-88ff-4330-a9d8-946e5231d1a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:24:14.120039 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9c5e2-88ff-4330-a9d8-946e5231d1a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81578) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:24:14.121721 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.fernet_utils [None req-58c9c5e2-88ff-4330-a9d8-946e5231d1a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81578) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:24:14.132600 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9c5e2-88ff-4330-a9d8-946e5231d1a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:24:14.134836 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9c5e2-88ff-4330-a9d8-946e5231d1a1 service placement] RBAC: Authorization granted {{(pid=81578) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:24:14.145837 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: [pid: 81578|app: 0|req: 1460/2937] 10.222.0.72 () {60 vars in 1365 bytes} [Tue Sep 13 13:24:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 13:24:24.141017 dv-cinder-801354-10-iscsi devstack@keystone.service[81578]: DEBUG dbcounter [-] [81578] Writing DB stats keystone:SELECT=89 {{(pid=81578) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}